Experience

• 

  Exposys data labs

  Apr 2020 - May 2020

  Web development intern
• 

  Tata consultancy services

  Mar 2021 - May 2021

  Software engineer intern

  Sperheaded the cleaning and transformation of over 1 million sales records using PySpark, ensuring impeccable data quality and integrity.Leveraged cutting-edge PySpark and Spark SQL for a thorough analysis of sales data, extracting valuable and actionable insights.Improved data accuracy by 25% through precise execution of data cleaning tasks, addressing missing values and outliers.Implemented broadcast join in PySpark, boosting data processing efficiency by 20%

• 

  Tata consultancy services

  Aug 2021 - Apr 2024

  Data engineer

  1.Orchestrated end-to-end Azure Data Factory (ADF) pipelines, accelerating SLG Report generation with a 30% reduction in data processing time.2.Implemented efficient data processing techniques, such as broadcast join and strategic file format selection (Parquet for reading data and CSV for writing final feed), resulting in a 40% reduction in report generation time.3.Collaborated cross-functionally to identify and rectify data anomalies, applying PySpark transformations to achieve a 70% accuracy improvement in data quality.4.Implementing Data Modeling techniques, specifically focusing on creating and managing SCD Type 2 structures5.Orchestrated the creation of an architectural design in collaboration with the Project Manager, aligning with the business specifications6.Supervised the Impact Analysis of changes in downstream systems, ensuring a smooth transition to Data Warehousing and Reporting Systems.7.Extracted data from the Oracle Database and generated mappings, workflows, and transformations to establish efficient pipelines for data transfer and increasing the sales by 27%.8.Developed and optimized SQL scripts to enhance the performance of workflows, maximizing efficiency in data processing and enhancing the performance by 30%9.Implemented Snowflake's Time Travel function in SQL queries, resulting in a 25% reduction in data analysis time, to scrutinize new data against historical records, driving insightful comparisons. Show less

• 

  Thoughtworks

  Apr 2024 - now

  Data engineer