Paawan Angra

Experience

• 

  Tata Power-DDL

  May 2019 - Jun 2019

  Intern
• 

  Wipro Limited

  Sept 2020 - Dec 2023

  Project Engineer

  Humana --Setup a risk reporting tool and got familiar with the Linux terminal.-Created ETL pipelines to connect streaming data from Kafka to Archer reporting tool for business owners.-Created ETL pipelines to load data from Kafka sources to MongoDB destinations.-Created pipelines to replace old .NET jobs leading to 75% reduction in runtimes.-Wrote python scripts to convert XML data into JSON format by applying business rules. Created pipelines to convert the JSON files to Parquet and load them into ADLS servers leading to 78% reduction in storage space compared to original data.-Tech used - Streamsets, Azure Data Lake, SQL Server, PythonFirst Horizon and Iberia Bank merger --As part of the migration created ETL pipelines using streamsets to merge data from on-prem databases of both banks like SQL, DB2 and Oracle to a common Azure Data Lake.-Converted the data to parquet files which saved 78% in storage.-Created Hive tables over the parquet data for easy access to business owners.-Created Oozie workflows to automate the whole process.-Tech Used - Streamsets. DB2, SQL, Oracle Servers, Azure Data Lake, Hive, Oozie Show less

• 

  Sigmoid

  Dec 2023 - now

  Software Development Engineer 2