Sai Kamal Kumar Kadarla

Experience

• 

  Modak

  Apr 2021 - now

  Software Development Engineer

  • Designed and implemented data movement workflows facilitating seamless data migration and transformation across diverse data sources by employing dynamically generated SQL and Scala code based on source and target data sources.• Developed highly efficient Java and SQL code for metadata crawling and centralized storage, empowering data analysis and management.• Enhanced data movement pipelines with object-level advanced options using SQL and Scala Spark, where users can perform transformations while ingesting data from one source to another. These options include parallel hash ingestion, data sorting, duplicate removal, and comprehensive column-level modifications (renaming, reordering, datatype changes, and adding new columns).• Significantly contributed to pipeline-level advanced options using Java and SQL, allowing users to execute additional tasks along with moving data between sources.(Pre/postconditions, target relative paths, archiving source raw data, and managing schema drift)• Implemented an advanced data connection feature using Java that allows users to leverage JDBC URLs for crawling metadata of relational data sources.• Developed robust post-condition functionality using Java and SQL that enables users to provide APIs that are automatically triggered in response to pipeline or data connection statuses.• Created workflow orchestration through Java that reduces manual efforts and give users to run each component of the product sequentially or concurrently according to their requirements.• Implemented the Variables feature using Java and SQL. This feature simplifies pipeline migration between environments by allowing users to define variables at an environment level and each component level.• Provided metadata insights for data analysis using SQL. These insights include total, new, and modified object counts as well as schema modifications of objects that occur after each crawl. Show less