Surya Gavini

Experience

• 

  Commerzbank

  Jan 2017 - Jan 2020

  Trainee Data Enginneer

  Stakeholder Engagement & Project Management: Actively engaged in regular requirement gathering and technical meetings, playing a crucial role in defect review and walkthrough discussions to align project objectives with stakeholder expectations throughout the project lifecycle.Data Pipeline Development: Expertly crafted data pipelines in Azure Data Factory (ADF) and Databricks, employing Integration RunTimes, linked services, and datasets. Utilized Notebooks to facilitate efficient data extraction and transformation from diverse sources, including Azure Blob Storage and on-premises SQL Data Warehouse.Advanced Analytics & Dataflow Creation: Spearheaded the development of sophisticated pipelines and dataflows for enhanced analytics. Focused on accessing, cleaning, and blending data from various sources using Azure Data Factory and Databricks to optimize data utility.Data Migration & System Integration: Successfully migrated on-premises SQL Server databases to Azure SQL Database. Innovatively moved SSIS packages into ADF using a lift-and-shift approach, subsequently transitioning from SSIS projects to more integrated solutions utilizing Azure Data Factory, Databricks, and Python.Dashboard Development & Data Visualization: Designed and built engaging storytelling dashboards in Power BI and Tableau. These dashboards provided critical insights into operational efficiency, enabling data-driven decision-making.Report Management & Security Implementation: Published comprehensive reports to Power BI Services, adeptly managing user access and groups. Implemented scheduled reports through Gateways and enforced Row-Level Security (RLS) to ensure data integrity and confidentiality. Show less

• 

  Data Artisans

  Aug 2021 - Apr 2022

  Data Engineer

  Stakeholder Collaboration & Project Implementation: Fostered strong collaborative relationships with stakeholders across various departments. Focused on designing, building, and deploying a range of initiatives on the data platform, ensuring alignment with cross-functional goals.Data Consolidation & Management: Skillfully consolidated large and complex datasets using Azure Data Factory, Databricks, Python, and SQL procedures. Efficiently migrated these datasets into Azure SQL databases, addressing both functional and non-functional business requirements.Pipeline Automation & Efficiency Enhancement: Played a pivotal role in automating data pipelines, significantly reducing workload and enhancing the efficiency of data quality and data load processes.Data Quality & System Reliability Improvement: Enhanced data quality, reliability, and overall efficiency of the database system. Achieved this by developing intuitive pipelines using Azure Data Factory, Databricks, Python, and SQL, which streamlined the data handling processes.Dashboard Development & Data Visualization: Created compelling storytelling dashboards using Power BI and Tableau, delivering actionable insights and fostering improved decision-making regarding operational efficiency. Show less

• 

  SDG Group UK

  Jun 2022 - now

  Data Consultant

  Python & Data Integration Expertise: Proficient in Python for developing robust data integration workflows and ETL processes, with a focus on efficient and effective data manipulation.Azure Data Factory & Data Pipeline Management: Skilled in using Azure Data Factory to orchestrate data pipelines, ensuring optimized data movement and transformation.Snowflake Cloud Administration & Snowpark: Experienced in Snowflake Cloud Administration, utilizing Snowpark for custom data processing and advanced analytics on Snowflake's cloud platform.Databricks for Analytics & Big Data: Proficient in Databricks for handling advanced analytics and large-scale data processing, using PySpark and SQL.Matillion Integration & Transformation: Knowledgeable in using Matillion for cloud-based data integration and transformation workflows.Power BI & Data Visualization: Adept at creating interactive dashboards and reports in Power BI, visualizing complex data insights effectively.SQL Database Proficiency: Strong foundation in SQL, with expertise in writing efficient queries and optimizing data retrieval.ETL Troubleshooting & Data Quality Assurance: Skilled in resolving issues related to data integration and ETL processes, focusing on maintaining high data quality.Data Modeling & Efficient Schema Design: Familiar with data modeling and schema design principles for effective data storage and retrieval.Problem-Solving & Detail-Oriented Approach: Excellent at solving complex problems and paying close attention to detail to ensure accurate data processing.Clear Communication Skills: Strong in conveying technical concepts to diverse audiences, including non-technical stakeholders. Show less