Experience

• 

  Itcrats

  Apr 2022 - Jan 2024

  Connected with clients daily to discuss project progress and updates.Worked within an Agile process to deliver projects in a timely and efficient manner.Performed data validation and ensure data accuracy and completeness by creating automated tests and implementing data validation processes.Analysing the source data.Transformed SQL queries into Spark SQL code for ETL pipelines.Rewritten Mainframe jobs into PySpark jobs using DataFrame API to process data and perform transformations.Developed ETL workflows in PySpark to extract data from various sources like data lake,SQL Server and transform it to meet business needs.Troubleshooting problems and resolving issues.Designed and developed Airflow DAGs to schedule and manage ETL workflows.Running Spark jobs on AWS Glue using Airflow DAGs.Used Hive for querying and reconciliation.Involved in Technical and Test case documents preparations. Show less

  • Associate ( AWS Data Engineer)

    Apr 2022 - Jan 2024

  • Assosiate Trainee ( Data Engineer/ETL Informatica Developer)

    Apr 2022 - Nov 2022