Bala Santhosh

Experience

• 

  Accenture in India

  May 2021 - now

  1. Demonstrated Expertise in Databricks: - Optimized schemas, tables, and permissions using general-purpose clusters, enhancing efficiency in data processing.2. AWS Services Proficiency: - Managed AWS services such as Secrets Manager, IAM roles, and policies, with hands-on experience in S3, Glue, and Athena for robust data storage and retrieval.3. Data Analysis Advancements with PySpark: - Implemented External Views on S3 files and created tables from parquet files using PySpark, elevating capabilities in advanced data analysis.Developed unit testing script for Data check and Count check using Pyspark 4. Efficient SLA Reduction: - Successfully reduced DAG SLAs by an impressive 70% through strategic task dependency optimization and utilization of general-purpose clusters.5. Python Scripting for Infrastructure Agility: - Developed and optimized Python scripts for cluster management in Databricks, enhancing infrastructure agility and responsiveness.6. Airflow DAG Optimization: - Created Airflow DAGs for JDBC Oracle to S3 and S3 to Databricks table orchestration, incorporating parallel tasks for improved operational efficiency.7. Effective Load Strategies with Airflow: - Implemented full and delta load strategies based on project requirements using Airflow Variables, ensuring seamless data synchronization.8. Version Control and Collaboration: - Maintained a version-controlled repository on GitHub for ELT processes, facilitating smooth collaboration and code management within the team.9. CI/CD Pipeline Establishment: - Established Artifact and Dataset pipelines for ELT activities in Jenkins, diligently monitoring and promptly resolving any CI/CD pipeline issues. Show less Developed and managed billing processes for a leading telecom company,focusing on product and discount data. Executed ETL operations to transfer dataseamlessly between sources.Worked on Spark SQL for data transformations (used joins, partitions , updatedand inserted new entries)Developed SQL queries for collecting the source data and migrated the data todifferent storage space.Working experience with Agile methodologies.Successfully completed many user-stories which have High Impact on projectSKILLS:- SQL, Excel, Streamsets, BRM, GIT, UNIX, Agile, JIRA Show less

  • Data Engineering Analyst

    Nov 2022 - now

  • Application Development Associate

    May 2021 - Nov 2022