PIYUSH JOSHI

Experience

• 

  Infosys Technologies Ltd

  Dec 2005 - Sept 2013

  Technology Analyst
• 

  Tata Consultancy Services

  Oct 2013 - Apr 2019

  > Designed data flows between Hive and AWS S3 storage system and Amazon RedShift> Create table schemas for Hive staging and intermediate environment.> Implemented logic to handle data related issues with python> Worked on designing and implementing Spark (pyspark) scripts for data ingestion and data processing using Spark APIs> Worked on optimization of pyspark code to improve performance of application> Worked on setting up multiple Cloudera hadoop clusters using multiple versions of CDHnamely CDH5.3(3 node cluster), CDH 5.7(5 node cluster), CDH 5.9 cluster (8 node cluster)and CDH5.14 (8 node cluster)> Worked on implementing Kerberos and enabling sentry on CDH 5.9 which is an 8 node cluster> Implemented Access Control for users on hive and hdfs using sentry commands.> Completed edx machine learning course on machine learning using Spark Mllib,Spark SQL APIs for solving a classification problem. > Have good hands-on knowledge of Spark RDD and Dataframes.> I have a good hands-on knowledge on Java and python Show less > Completed "TGS Salt Identification Challenge" in kaggle and achieved a top 35% percentile. Used Keras and tensorflow for solving this image segmentation problem.> Implemented a "Click through Rate prediction" using Spark APIs.> Have a good understanding and hands on experience on multiple CNN architectures as well as RNN, LSTM.> Design and develop a user based recommender system using Mahout and also run predictive analytics on the data.> Created a number-plate recognition application using opencv and tesseract python libraries. Show less

  • Project Leader

    Oct 2013 - Apr 2019

  • Machine Learning Specialist

    Oct 2013 - Apr 2019
• 

  Wells Fargo

  Apr 2019 - now

  Technical Lead/ Manager