Pawankumar Manpuri

Experience

• 

  Go Digital Technology Consulting LLP

  Sept 2021 - Nov 2022

  Data Engineer

  Experience:Designed and maintained optimal data pipelines using AWS S3, EMR, Athena, Spark, and SQL for efficient ETL processes from diverse sources.Successfully migrated on-premises data solutions to AWS, including data validation and enhancements to ensure a seamless transition and improved data integrity.Enhanced data quality and reliability through comprehensive validation, cleansing, and enrichment techniques.Developed and supported Hadoop, ETL, and SQL solutions for robust data processing and analysis.Identified and implemented internal process improvements by automating tasks, optimizing data delivery, and building scalable infrastructure with AWS services. Show less

• 

  Vertoz

  Nov 2022 - Feb 2025

  Data Engineer - II

  Experience:Implemented efficient end-to-end PySpark ETL workflows, integrating Kafka data into HDFS, facilitating efficient data processing in Vertica.Efficiently handled terabytes of incoming data daily, optimizing data pipelines to meet real-time needs and reducing latency by 50%.Increased real-time data processing by 800% from 100,000 requests per second (RPS) to 900,000 RPS using Spark Streaming on Kafka, significantly improving analytics speed and efficiency.Optimized API data fetching for diverse formats, including JSON, CSV, and XML, reducing integration time from several days to near one-click (90% reduction in time).Automated MIS reports by integrating data from Google Ads, Facebook Ads, and other sources for comprehensive analysis and decision-making reducing manual effort by 70%.Successfully migrated Scala/Java based data processing jobs to Python/PySpark, enhancing code maintainability and reduced development costs. Show less

• 

  Deloitte

  Mar 2025 - now

  Consultant