Experience

• 

  Cognizant

  Mar 2022 - Jul 2022

  Big data intern

  ◦ Completed Big Data Engineering Training covering core concepts of distributed systems, data pipelines and data warehousing.

• 

  Cognizant

  Oct 2022 - now

  Data engineer

  ◦ Engineered and deployed ETL pipelines in Azure Databricks to process and transform structured datasets of up to 30 gigabytes daily, supporting data analytics and reporting needs for Clients Big Data platform.◦ Automated ingestion workflows using Pyspark and Python, ensuring seamless integration with various data sources such as SQL databases, and Delimited files.Achieved a 40% reduction in manual data processing time◦ Developed and maintained lakehouse architecture using delta tables to ensure high data reliability and efficient data versioning to further support all the business user needs.◦ Developed high-performance SQL queries,streamlining data extraction processes and enhancing query efficiency by 35%, while enabling deeper analytical insights from refined datasets.◦ Configured and monitored Azure Databricks pipelines for scheduled batch data processing and Incremental data synchronization.◦ Utilized Databricks Notebooks for collaborative data exploration, visualization, and debugging of complex workflows.◦ Integrated Azure Data Lake Storage storing over 5 terabytes for scalable data storage solutions, ensuring high availability and cost-efficiency.◦ Supported 2+ business critical power bi dashboards by ensuring high data availability and consistency.◦ Adhered to CI/CD practices for data pipelines, leveraging tools like Git and Azure Devops for version control and deployment. Show less