Lingeswaran M

Experience

• 

  Scope eKnowledge Center

  Apr 2013 - Apr 2014

  Web Analyst

  Executing SQL statement in DB2 and POSTGRE SQL databases, provided by the Application Team.Running scripts for Backup and Restore and troubleshooting in case of failures/issues.Performing Bulk load of the data into the client's web portal databases.Monitoring and testing the front end of the data values in HTML page.

• 

  IBM India Pvt Ltd

  Apr 2014 - Feb 2017

  Senior Database Administrator

  DB2 DBA SkillsWorked on DPF size ranges upto 17TB and range partitioned tables.Restoration of databases from online and offline backups.Performed Cross-Node Recovery completely by Shell scripts.backups and jobs, troubleshoot backup failures etc.Installing Version and FixPack Upgrades of DB2.Cataloging remote and loop-back databases based on customer request.Responsible for creating Databases, table spaces, tables

• 

  Wipro Limited

  Feb 2017 - Sept 2017

  Senior Database Administrator
• 

  Tata Consultancy Services

  Sept 2017 - Jun 2019

  IT Analyst

  Developed the code to generate the DDL queries to create the tables in Hive based on the technical metadata.Extensively worked on creating partitions, buckets, Performance tuning, Compression-related properties in Hive.Writing Shell scripts & scheduling cron jobs for loading, cleansing and preparing the data before it is moved to Hadoop Data Lake.Gathering Business requirements and preparing detailed specifications that follow project guidelines which are required to develop programs.Customer interactions on a daily basis – for Knowledge Transfer, status reporting, clarifications. Show less

• 

  Altimetrik

  Jun 2019 - Nov 2023

  Data Engineer

  Orchestrated end-to-end data ingestion from electric/hybrid car sensors to HDFS/Hive through Spark,tailoring processes to meet specific requirements.Translated Hive Queries into diverse Spark (Python) Actions and Transformations, creating RDDs and Dataframes from essential files in HDFS.Constructed Kafka/Nifi pipelines to seamlessly transport data from the cloud to on-premise servers.Engineered comprehensive Hive Queries for parsing raw data, populating external and internal tables,and storing refined data in partitioned external tables. Additionally, executed the complete conversion of outdated Java-based map reduce code to PySpark. Show less

• 

  Publicis Sapient

  Nov 2021 - now

  Senior Cloud Data Engineer

  Senior Associate Data Engineering L2 -Presently working with Publicis Sapient, Global Markets since Nov, 2021. Publicis Sapient Global Markets is a leading provider of services to today's financial and commodity markets, providing a full range of capabilities to help clients grow and enhance their businesses, Designing and delivering high-impact solutions for the opportunities and issues of today's dynamic global marketplace.Roles and Responsibilities: Creating data pipeline with Kafka - BigQuery integration, data ingestion, transformation using DBT - defining and implementing data models, partitioning, clustering and optimizing performance of BigQuery.Designing new Database architecture for new applications from scratch on Cloud Spanner.Creating POC and documenting in Confluence about Utilization, Capacity, Backup and Restore, Pricing, Auto-Scaling etc. Show less