Experience

• 

  Headrun

  Nov 2012 - Oct 2016

  Product - Bionco• Proactively improved system performance through adjustments and bug resolution, enhancing overall operational efficiency.• Designed and implemented a robust data processing pipeline, streamlining data ingestion into databases for efficient storage and retrieval.• Implemented data de-duplication using Python and MySQL, leveraging the dedupe module to ensure data integrity and accuracy.• Successfully managed data uploads to Algolia and developed search applications, enabling seamless and efficient search functionality.• Optimized data loading time through the implementation of multi-threading techniques, enhancing system responsiveness and user experience.• Played a vital role in building services using Python, utilising the in-house funcserver framework to deliver high-quality and efficient solutions. Show less Products - rqst & User Profiles• Played a pivotal role in writing highly maintainable and efficient code in Python and Go for the rqst system, responsible for crawling 200 million RSS feeds daily.• Led the development of a robust Public Proxy System, maintaining a pool of public proxies to enhance system capabilities.• Contributed significantly to the enhancement of the in-house Scrapy-like scraper framework, optimizing data crawling processes.• Implemented comprehensive monitoring solutions, including writing metrics to Graphite using StatsD and building informative Grafana dashboards.• Managed a 10-node crawling cluster deployed with a master-slave architecture, ensuring reliable and efficient data extraction.• Established and maintained a multi-node ElasticSearch cluster, enabling efficient search capabilities across the platform.• Took charge of building the search backend application using the Django framework, delivering seamless and powerful search functionality.• Played a key role in the recruitment and training of freshers, contributing to team growth and development. Show less Product - Buzzinga• Developed specialized crawlers utilizing Python Scrapy platform, enabling efficient and targeted data extraction.• Played a key role in collecting RSS feeds from various sources on the internet through the creation of Python scripts, contributing to a comprehensive data repository.• Optimized the efficiency of RSS feed extraction from Google Blogspot by implementing a location hierarchy, enhancing data retrieval capabilities.• Demonstrated proficiency in generating daily and weekly system health reports, providing critical insights into system performance, index growth, API performance, and downtime.• Contributed to the development of APIs using the Django framework, facilitating seamless data exchange and functionality.• Ensured application reliability by writing and executing test cases utilizing the Selenium framework, validating the robustness and accuracy of the software. Show less

  • Senior Software Engineer

    Jan 2016 - Oct 2016

  • Software Engineer

    Aug 2014 - Jan 2016

  • Associate Software Engineer

    Nov 2012 - Aug 2014
• 

  Deepcompute

  Oct 2016 - Feb 2023

  Product - nferX(Clinical Trials, DrugCatcher, etc..)Data engineering• Spearheaded the design and implementation of robust data engineering pipelines, with a focus on mapping drugs to various entities including targets, companies, diseases, and more.• Led the design and development of a sophisticated engineering pipeline, which involved processing daily scraped news articles and executing multiple machine learning models. This initiative significantly contributed to the discovery of new drugs and companies in the research stage.• Orchestrated the end-to-end development of the Instruments app, overseeing a dedicated team of four members. This encompassed tasks such as data crawling, analysis, API design and development, as well as hosting the application on a production platform. Additionally, took charge of continuous status tracking and agile release planning.Data crawling• Managed a 20-member team specializing in data crawling, centralizing data from various public websites.• Implemented monitoring tools (Grafana, Kibana, custom solutions) for critical metric tracking.• Developed the Crawl Stats API/UI, a centralized source for hundreds of indexed websites.• Led real-time COVID-19 case data extraction from all US states, supporting app development. Motivated the team to meet tight deadlines.• Extracted drug revenue data from complex SEC XBRL filings.• Delivered structured data for Clinical Trials, Proteomics, Adverse Events, Small Molecules, etc. Show less Product - nferX• Spearheaded the development of reusable backend components and tools, enhancing efficiency and productivity across services.• Contributed significantly to the development of various services tailored for the biomedical client, nference.• Successfully architected and implemented a multi-node Sharded MongoDB cluster, providing a scalable and reliable database solution for cross-team utilization.• Designed and implemented a robust multi-node ElasticSearch cluster, enabling efficient search functionality for millions of documents.• Played a key role in the design and development of critical services including Vocab Explorer, Token Collection, More Info, and more.• Led the creation of an SDK, empowering customers to seamlessly interact with our APIs, enhancing accessibility and usability.• Managed a data crawling team, specializing in web analysis and Python parser development for structured/semi-structured data extraction. Show less

  • Engineering Manager

    Apr 2019 - Feb 2023

  • Engineering Team Lead

    Oct 2016 - Mar 2019
• 

  Nference

  Mar 2023 - now

  Engineering manager

  Product Neuviz• Led the development and launch of neuviz, a cutting-edge 3D visualization mapping software utilized by EP (Electrophysiology) teams as a retrospective tool to visualize procedures conducted in the cathlab.• Successfully collaborated with the SRE (Site Reliability Engineering) team to deploy and manage a robust GCP (Google Cloud Platform) cluster tailored for the project's requirements.• Orchestrated a cross-functional team effort, resulting in the phased development of the application, enhancing functionality and user experience in two distinct releases.• Spearheaded collaboration with the datagen team, resulting in the implementation of a Kubernetes solution for seamless initiation of a Kubernetes job each time a case was uploaded. Show less