Sivaram Sundar

Experience

• 

  Cognizant

  Aug 2021 - Oct 2022

  • Programmer Analyst

    Aug 2022 - Oct 2022

  • Programmer Analyst Trainee

    Aug 2021 - Aug 2022
• 

  Unify Technologies

  Oct 2022 - now

  Software Development Engineer ( Data Engineer )

  ⬛ Client: Apple Inc⬛ Working as a Data Engineer in Apple Advertisements Platform Engineering Team➡️ Developed scalable and optimized data pipelines which deals with a data size of 3 TB+/day using Spark scheduled with Apache Airflow➡️ Built robust and reliable data pipelines on AWS using Apache Spark and transformed data based on various data quality check rules➡️ Developed Automation frameworks using Spark to automate Data Quality Checks in various data pipelines, created a DQC pipeline out of the DQC Automation framework using Jenkins CI/CD pipeline➡️ Extended the development of a Data Generation framework using Java, Docker, Kubernetes and shell scripts that push data to Kafka and AWS S3 in Avro format➡️ Developed Data quality Automation frameworks using Spark and Hive to automate the entire data validation process for data flowing through various data pipelines, hence eliminating all the manual data validation effort which resulted in huge time & cost saving ➡️ Worked on converting MapReduce Jobs scheduled on Oozie to Spark-Scala Jobs scheduled on Airflow ➡️ Optimized the data pipelines dealing with 3 TB+/day data by reducing the runtime by 30% ➡️ Worked on solving Complex Data Transformation Usecases and Data Cleaning using UDF's➡️ Was part of the complete software development life cycle, which includes development, testing and debugging➡️ Worked with cross functional teams and product teams for requirement gathering and analysis Show less