Kaushik Ghosh

Experience

• 

  Tata Consultancy Services

  Feb 2020 - Sept 2022

  • Developing new pipeline solution for client to fetch data from servers and transform according to the requirements.• Modify and debug existing manual solutions into automation and less time and resource consuming solution.• Report creation for client according to the request on a daily and monthly basis.• Automation job monitoring - Restart and Force Complete failed job and manual completed job.• Manage, alter testing ETL/ELT pipeline on an occurrence of job failure.• Cross-functional team collaboration to develop open-source software solution using python to reduce software license cost.• On-boarding new associates to team and providing training with all required technology and process required by the project. Show less

  • System Engineer

    Feb 2022 - Sept 2022

  • Assistant System Engineer

    Feb 2020 - Jan 2022
• 

  PwC India

  Sept 2022 - Aug 2024

  Consultant

  • Led the design and implementation of a modern data architecture on Azure using Data Factory, Databricks and Synapse, enabling the ingestion, storage, and analysis of petabytes of data from diverse sources, resulting in improved data accessibility and faster insights for business stakeholders. • Developed and optimized ETL pipelines to streamline data integration processes, reducing data latency by 30% and improving data accuracy by implementing advanced data cleansing and validation techniques. • Designed and implemented a scalable data lake solution on Azure Blob Storage, leveraging Azure Data Lake Storage Gen2 and Azure Synapse Analytics to enable ad-hoc querying and analytics on semi-structured and unstructured data. • Collaborated with cross-functional teams to understand business requirements and translate them into technical solutions, ensuring alignment with strategic objectives and delivering value-added insights to key stakeholders. • Implemented robust data security measures, including encryption at rest and in transit, role-based access control, and data masking, to protect sensitive information and ensure compliance with industry regulations. • Proactively monitored system performance and implemented performance tuning strategies to optimize resource utilization and minimize costs, resulting in a 20% reduction in Azure service expenditures. Show less

• 

  Accenture

  Aug 2024 - now

  Senior Data Engineer (Package App Development Senior Analyst)

  Senior Data Engineer