Vivek Singh Rajput

Experience

• 

  Infosys

  Jul 2014 - Oct 2017

  • Developed ETL programs using Informatica to implement the business requirements. Communicated with clients to discuss the issues and requirements. • Effectively worked on Onsite and Offshore work model. Involved in production support to resolve the ongoing issues and troubleshoot the problems after production deployment.• Effectively used relational SQL wherever possible to minimize the data transfer over the network, used Informatica parameter files for defining mapping variables, workflow variables, FTP connections and relational connections, debugger in identifying bugs in existing mappings by analyzing data flow. • Involved in enhancements and maintenance activities of the data warehouse including tuning, code enhancements, performance tuning at the functional level and map level. • Reviewed and analyzed the functional requirements, mapping documents, problem solving and trouble shooting. Performed unit testing at various levels of ETL and actively involved in team code reviews. • Identified problems in existing production data and developed one time scripts to correct them. • Fixed the invalid mappings and troubleshoot the technical problems of the database. Show less Involved in all phases of SDLC from requirement gathering, design, development, testing, Production, user training to support for production environment.Created mappings and extracted data from various sources, transformed data according to the requirement. Prepared Technical Design documents and Test cases.Performed data manipulations using various Informatica Transformations like Filter, Expression, Lookup (Connected and Un-Connected), Aggregate, Update Strategy, Normalizer, Joiner, Router, Sorter and Union.Implemented slowly changing dimension methodology for accessing the full history of accounts.Involved in internal and external reviews as well as formal walk through among various teams and documenting the proceedings. Show less

  • Senior System Engineer

    Jul 2016 - Oct 2017

  • System Engineer

    Jul 2014 - Jun 2016
• 

  LTI - Larsen & Toubro Infotech

  Nov 2017 - Sept 2021

  Senior Data Engineer

  • Estimate the development effort required to replicate existing Informatica mappings in Hadoop. Understand the logic behind these mappings and plan the migration process.• Translate the logic from Informatica mappings into SQL queries that can be executed using Spark-SQL and DataFrames. Ensure data consistency and accuracy during the conversion process.• Create and manage Hive tables and views in the functional layer. Optimize data storage and retrieval for efficient querying.• Write shell scripts to automate the execution of Spark jobs. Monitor job performance and troubleshoot any issues.• Prepare detailed deployment documents for the developed application. Facilitate knowledge transfer to the Production support team.• Coordinate the deployment schedule for the Hadoop application in both Pre-production and Production environments. Show less

• 

  IBM

  Sept 2021 - now

  Senior Data Engineer

  • Developed and maintained data pipelines using Spark and Pyspark to ingest consented digital marketing acquisition events data from diverse sources, ensuring seamless data integration.• Applied SparkSQL and Hive for data transformation, aggregation, and joining operations to enrich the raw event data with customer demographics and transactional details.• Employed advanced analytics techniques with Spark to identify the most influential digital marketing channels and events in driving customer acquisition, leveraging concepts like attribution modeling.• Facilitated data governance policies and security measures to safeguard sensitive customer information and ensure compliance with regulatory standards like GDPR and CCPA.• Developed interactive dashboards and visualizations using Kibana to present key metrics and insights, facilitating data-driven decision-making for marketing strategy optimization.• Conducted performance tuning and optimization of Spark jobs and Hive queries to enhance processing efficiency and reduce latency, ensuring timely delivery of insights to stakeholders.• Contributed to the continuous improvement of data quality and accuracy by implementing data validation and cleansing processes, enhancing the reliability of insights derived from the digital marketing campaign data.• Co-ordinated with cross-functional teams, fostering collaboration and knowledge sharing to achieve project objectives effectively. Show less