Lucas Gomes

Lucas Gomes

Data Engineer Intern

Followers of Lucas Gomes456 followers
location of Lucas GomesBrazil

Connect with Lucas Gomes to Send Message

Connect

Connect with Lucas Gomes to Send Message

Connect
  • Timeline

  • About me

    Data Engineer | Azure | AWS | Pyspark | Airflow | SQL Server

  • Education

    • Centro Federal de Educação Tecnológica de Minas Gerais

      2013 - 2016
      Técnico em Informática Computer Science
    • PUC Minas

      2020 - 2025
      Graduação Sistemas de Informação
    • Universidade Federal de Juiz de Fora

      2017 - 2019
      Bacharelado em Engenharia Electrical and Electronics Engineering

      Curso interrompido.

  • Experience

    • Thermon Mechanical Industry

      Aug 2019 - Feb 2020
      Data Engineer Intern

      Experience in network management, database management, IT resource maintenance, and VBA development for Microsoft Access applications with tools for environmental and quality management control. Additionally, I was responsible for generating reports using SQL queries on the company's database

    • Dti digital

      Mar 2021 - Jul 2022

      Participated in the architectural decisions and implementation of a comprehensive data lake infrastructure within AWS. By utilizing AWS Lambda, EMR with boto3, S3, Athena and Dremio. These resources were implemented using Terraform, ensuring consistent and automated provisioning. In this role, I worked mostly with two clouds: GCP and Azure. On GCP, I created pipelines with Apache Beam to process and clean data, feeding a Data Lake in Cloud Storage and BigQuery. I also developed data analysis and visualization using Data Studio. On Azure, I used Azure Resource Manager for IaC and CI/CD to create data pipelines with Synapse Dataflow and Apache Spark Pools. I actively contributed to dataOps processes.

      • Data Engineer

        Apr 2022 - Jul 2022
      • Data Engineer Trainee

        Mar 2021 - Apr 2022
    • A3Data

      Jul 2022 - Nov 2023
      Data Engineer

      Creation and maintenance of a Data Lake on Azure, consuming data from various sources such as Athena, SQL Server, and media platforms like Google ADS, Google Analytics, Facebook ADS, and TikTok ADS. This was achieved through the use of APIs, Data Factory integrations, and the Airbyte tool. The data was processed using Data Factory and clusters powered by PySpark provisioned through Kubernetes. All of this was orchestrated by Airflow.

    • Inter

      Nov 2023 - now
      Data Engineer
  • Licenses & Certifications

    • 25ª META - Mostra Específica de Trabalhos e Aplicações

      Centro Federal de Educação Tecnológica de Minas Gerais
      Oct 2015
      View certificate certificate
    • Academy Accreditation - Databricks Lakehouse Fundamentals

      Databricks
      Apr 2024
      View certificate certificate