Mohammad Arshad

Experience

• 

  Huawei Technologies India

  May 2007 - May 2022

  Lead Zookeeper team of 4 members. Upgraded ZooKeeper version to 3.5.6, added/re-implemented private features, supported integration and contributed few feature in open source.ContributedLead the work to automate hbase cluster repair through HBCK2 command to make maintenance personals job easy.Added new commands in HBCK2 to list and fix hbase cluster problems. Fixed existing hbck2 issues and contributed to open source.Took ownership of RSGroup feature, verified its functionality, fixed existing RSGroup issues and contributed to open source. Show less Core member of team that designed and implemented new architecture of hbase secondary index.Initial design code is available at https://github.com/Huawei-Hadoop/hindex)Implemented secondary index handling in following features - Region split: rebuild the index after split- Snapshot: include index information in snapshot, create index when restored or cloned- Compaction: delete index data when user data deleted, handle TTL, compaction and direct delete- Replication: don’t replicate index data- Bulk load: before loading HFile into cluster if region splits, rebuild the indexes- WAL Player: recreate the index region boundaries are changedKey member of Data Lake Center (based on HDFS and HDFS Router) TRP team.Implemented features and open sourced few HDFS-13443, HDFS-13495 Show less MRS (similar to Amazon EMR):As a supporting member worked few months on Huawei enterprise-level big data clusters on the cloud MRS. Worked on Manager, broker, API-Gateway and console etc. Done coding in go lang.ZooKeeper:Took end to end ownership of ZooKeeper. Analyzed, designed and implemented ZooKeeper requirements. Fixed issues, supported customers, handled CIHCC Demo:Lead a team of 3 members to develop Huawei BigData capability showcasing application which was presented in Huawei Connect Conference. Worked as technical lead, customer interface person, individual contributor. Show less

  • Lead Architect

    May 2021 - May 2022

  • Senior System Architect

    Dec 2019 - May 2021

  • System Architect

    Nov 2018 - Dec 2019

  • Senior Technical Lead

    Jul 2016 - Oct 2018

  • Technical Project Lead

    Aug 2013 - Jun 2016

  • Senior Software Engineer

    May 2010 - Jul 2013

  • Software Engineer

    Mar 2008 - Apr 2010

  • Associate Software Engineer

    May 2007 - Feb 2008
• 

  The Apache Software Foundation

  Jan 2017 - now

  https://zookeeper.apache.org/credits.html

  • Apache Ambari Commiter

    Aug 2023 - now

  • Apache ZooKeeper Committer

    Jan 2017 - now
• 

  Visa

  May 2022 - now

  Data Engineer - Senior Consultant level

  Key member of the engineering team that developed in-house Open-source Hadoop distribution and successfully replaced Hortonworks Data Platform (HDP).Worked on Ambari, Ambari Metrics, ZooKeeper, Sqoop, Bigtop etc. Handled backward incompatibility issues, fixed bugs, enhanced, remediated CVEs, supported in cluster upgrade. Delivered some of these works with assistance of a junior team member.Stabilized and enhanced Apache Ambari metrics for 2000 node Hadoop cluster. Contributed some of the Apache Ambari work to open source, became Apache Ambari committer. Show less