Diptarka Dutta

Experience

• 

  Tata Consultancy Services

  Apr 2019 - Jul 2021

  Associate System Engineer

  1. Pl SQL based project for ETL in Data Warehouse Project2. Oracle Apex Development to built Access-based Portal3. Unix to build for SQL loader and Data acquisition 4. Oracle Business Intelligence to build Reporting Tool

• 

  PwC India

  Aug 2021 - Nov 2022

  Associate

  -Developed dynamic, end-to-end pipelines in Azure Data Factory (ADF) to ingest data from multiple sources (CSV, API call, MongoDB, Cosmos DB, Excel) for 10+ use cases.– Utilized Azure Databricks (Pyspark) notebooks, dataflows, and stored procedures to transform complex datasets.– Got client appreciation for Optimizing pipelines and Spark jobs to improve performance and minimize load times.– Created Business Data Views (BDVs) by joining different data layers using Tibco Data Virtualization (TDV) as per thedata model.– Implemented query optimization and performance tuning techniques, accelerating the end-to-end process by 50%. Show less

• 

  EY

  Nov 2022 - now

  Senior Analyst

  Member of the AI and Data team, specializing in developing and deploying innovative solutions by integrating AI and data engineering principles.AI Engineering Expertise-Generative AI Systems: Designed and implemented systems leveraging large language models (LLMs) with frameworks like RAG, enhancing performance through PEFT techniques.-Applications: Developed advanced solutions such as chatbots and fine-tuned models for diverse use cases, delivering enhanced AI capabilities tailored to client needs.Data Engineering Expertise-Led a team to process and transform diverse data formats (CSV, JSON, Oracle DB) into Parquet using Azure Data Factory (ADF) pipelines, orchestrating key modules for Data Ingestion (DI), Data Quality (DQ), and Data Harmonization (DH).-Created a Data Quality (DQ) module in Azure Databricks (ADB), implementing validation, profiling, and cleansing processes, which reduced execution time by 33.3% for subsequent data modules.-Optimized PySpark jobs for processing complex JSON data on Dataproc and loaded data into BigQuery, improving job performance and efficiency. Also, streamlined job orchestration using Cloud Composer for smooth workflows.SectorsInsurance, Real estate, Oil and Gas, Banking Show less