Aravind Sriram

Experience

• 

  Modak

  Jul 2019 - Jan 2023

  Automated API Ingestion:Tools & Technologies : Azure Data factory, Azure function, Python, Azure Data Lake, Azure DevOps, Azure SQL databaseRoles & Responsibilities:1 Developed python Azure function to read configs from the user along with type of ingestion whether each response should be pushed directly to data lake or concatenated as single file or each response as single file2 Developed Azure Data factory pipeline to orchestrate the ingestion and load the data to SQL Server 3 Handled change data capture using Azure data flow4 Pipeline is configured and can be used for different API’s across teams for Automated ingestion into different tables which has reduced manual time by 10/hrs. month as the sources are increasing with same parsing logic’sSemi Structured Ingestion/Parser:Tools & Technologies: Python, Pyspark, PostgreSQL, Hive, Airflow, Azure DevOps, HDFSRoles & Responsibilities:1 Developed Pyspark code for automating XML ingestion into HDFS.2 Airflow DAG for pipeline Orchestration which triggers the downstream pipelines post process3 Developed XML parsers to load data to multiple tables from each XML basing on business logic4 Handled retry/rollback mechanism at DAG level and code level5 Reduced processing time from month to 3 days by implementing parallel processingAutomated Structured Ingestion:Tools & Technologies: Python, Pyspark, PostgreSQL, Azure Databricks, Azure DevOps, Azure SQL Database, Big QueryRoles & Responsibilities:1 Designed and Drafted Databricks note books for generic ingestion from different structured JDBC sources and Azure data factory for orchestration2 Reduced development cycle by ingesting data to different cloud providers SQL database (AZURE), Big Query (GCP) to provide data availability to Data Scientist across platforms to improve faster development and take advantages of respective tools in respective platforms Show less

  • Data Engineer

    Oct 2019 - Jan 2023

  • SDE Intern

    Jul 2019 - Sept 2019
• 

  HSBC

  Feb 2023 - now

  Senior Software Engineer (Data)