Sagadevan K

Experience

• 

  Quantiphi

  Jul 2021 - now

  Data Engineer

  Written an AWS glue job to load data from SQL-Server to Redshift Staging layer via S3. Another glue job to move data from Redshift Staging layer to Redshift ODS layer. Also worked on Unit tests and E2E tests using Docker and moto.Written Python scripts to load Demand Generation data from Google sheets to Amazon RDS MySQL instance with the help of GCP service accounts. Created an AWS QuickSight dashboard to visualise this data and gain insights from them.Written an AWS Glue job to load data from S3 Raw(CSV files brought by DMS) to Redshift staging layer after intermediate storage in S3 in parquet format. The process involves deduplication for CDC based DMS jobs, auditing the table level status to Dynamodb Show less