Nithin N

Experience

• 

  Magneto IT Solutions

  May 2014 - Jun 2015

  Hadoop Developer

  •Implemented solutions for ingesting data from various sources and processing the Data-at-Rest utilizing Bigdata technologies such as Hadoop, Map Reduce Frameworks, HBase, and Hive.•Involved in creating Hive Tables, loading with data, and writing Hive queries which will invoke and run Map Reduce jobs in the backend.•Used IMPALA to read, write, and query the Hadoop data in HDFS and configured KAFKA to read and write messages from external programs.•Expertise in implementing Spark Scala applications using higher-order functions for both batch and interactive analysis requirements. Show less

• 

  Walgreens Boots Alliance

  Aug 2016 - Aug 2018

  Big Data Engineer

  •Contributed to the development of key data integration and advanced analytics solutions leveraging Apache Hadoop and other big data technologies for leading organizations using major Hadoop Distributions like Hortonworks.•Responsible for data extraction and data integration from different data sources into Hadoop Data Lake by creating ETL pipelines Using Spark, MapReduce, Pig, and Hive.•Used IMPALA to read, write, and query the Hadoop data in HDFS and configured KAFKA to read and write messages from external programs.•Developed Spark programs with Scala and applied principles of functional programming to process complex unstructured and structured data sets. Processed the spark data with Spark from the Hadoop Distributed File System.•Developed ETL pipelines using notebooks, Spark Data frames, SPARK SQL, and Python scripting. Show less

• 

  JPMorganChase

  Aug 2018 - Sept 2022

  Senior Big Data Engineer

  • Created AWS Launch configurations based on customized AMI and used this launch configuration to configure auto scaling groups Implemented AWS solutions using EC2, S3, RDS, Route53, EBS, Elastic Load Balancer, and Auto scaling groups.• Loaded and transformed large sets of structured, semi-structured, and unstructured data using Hadoop/Big Data concepts.• Expert in implementing advanced procedures like text analytics and processing using in-memory computing capabilities like Apache Spark written in Scala. Expertized in implementing Spark using Scala and Spark SQL for faster testing and processing of data and responsible for managing data from different sources.• Responsible for data extraction and data integration from different data sources into Hadoop Data Lake by creating ETL pipelines Using Spark, MapReduce, Pig, and Hive.• Collaborated with Architects to design a Spark model for the existing MapReduce model and migrated them to Spark models using Scala. Worked on writing Scala Programs using Spark-SQL in performing aggregations. Show less

• 

  PayPal

  Sept 2022 - now

  Senior Big Data Engineer

  • Experience in transforming structured and unstructured data, setting up Hadoop clusters, and developing data pipelines using Pig, Sqoop, and Databricks. Proficient in AWS services for data integration, leveraging AWS EMR, and migrating on-premises code to the cloud.• Architected modern data solutions with Snowflake and developed ETL processes using AWS Glue and PL/SQL. Integrated Tableau with enterprise data visualization and real-time analysis standards, utilizing Tableau Desktop and Interworks Tableau workbook tools. Ensured data quality from source to target repositories and created interactive dashboards and reports.• Automated 95% of data processing workflows by developing robust Apache Airflow DAGs, leading to a 40% reduction in manual oversight and a 30% increase in process reliability.• Deployed scalable data warehousing solutions on AWS, integrating EC2, S3, Redshift, and Glue, which improved ETL processing speed by 35% and reduced storage costs by 25%. Show less