Shivanshu Tiwari

Experience

• 

  Wipro

  Jul 2019 - Aug 2020

  Data Engineer

  -Supported the design and optimization of ETL pipelines, reducing data processing time by 20%.-Partnered with team members to analyze business requirements and develop solutions for seamless data integration and transformation.-Assisted in the implementation of streaming solutions using Spark Streaming, enabling near real-time analytics.-Maintained high data accuracy and integrity by conducting thorough testing and troubleshooting of pipelines.

• 

  Celebal Technologies

  Jan 2022 - Mar 2024

  Data Engineer

  -Designed and built 10+ scalable data pipelines, enabling efficient processing and transformation of structured and unstructured data.-Developed and implemented data models and data warehousing solutions, improving query performance and data accessibility for end-users.-Spearheaded ETL processes to ensure the timely and accurate processing of datasets exceeding 10TB+ in size.-Collaborated with cross-functional teams to define data requirements and implement solutions, achieving 99.9% SLA adherence.-Implemented data cleansing and transformation pipelines, increasing data quality and consistency across systems. Show less

• 

  Koantek

  Mar 2024 - now

  Data Engineer

  -Led the migration of 100+ data assets to Unity Catalog, ensuring seamless integration and compliance with governance policies.-Transitioned Databricks notebooks and Delta tables to Unity Catalog, enhancing data security and lineage tracking.-Migrated and optimized Azure Data Factory (ADF) and Databricks workflows, improving efficiency by 30% and enabling better data governance.-Collaborated with cross-functional teams to ensure successful adoption of Unity Catalog, resulting in improved data governance and accessibility. Show less