Talk to sales Try for Free
Daniel S.

Daniel S.

Estagiário

Followers of Daniel S.1000 followers
location of Daniel S.Recife, Pernambuco, Brazil

Connect with Daniel S. to Send Message

Connect

Connect with Daniel S. to Send Message

Connect

  • Timeline

  • About me

    Senior Data Engineer | AWS | Azure | Spark

  • Education

    • Catholic University of Pernambuco

      2005 - 2008
      Computer Science

  • Experience

    • INNOVATIVE SOFTWARE

      Jul 2006 - Dec 2007
      Estagiário

      C#, ASP.NET, SQL

    • ICONN TECNOLOGIA

      Jan 2008 - Apr 2009
      Desenvolvedor de software

      C#, ASP.NET, SQL

    • Zênite Soluções em TI

      Apr 2009 - Sept 2010
      Business Intelligence Developer

      requirements gathering, BI, SSIS, SSAS, SQL, dashboards.

    • NeuroTech

      Sept 2010 - Nov 2011
      Business Intelligence Consultant

      SSIS, SSAS, SQL, SQL Server, Dashboards, T-SQL, Data Integration, ETL

    • MSBS Tridea

      Jan 2012 - Sept 2013
      Business Intelligence Specialist

      - Design and implement Data Warehouse solutions.- Create Data warehouses using Analysis Services (Tabular / Multi-dimensional). - Create BI Solution- Create integration solution using python- Build dashboards- Create ETL solutions using SQL (Stored Procedures)

    • Niteo Informática

      Sept 2013 - Jul 2014
      Business Intelligence Specialist

      working in the stock exchange BM&FBovespa.- Design and implement Data Warehouse architecture- Create Data warehouses using Analysis Services (Tabular). - Create BI Solution using Power BI- Create an integration solution using python- Build dashboards- Create ETL solutions using SQL (Stored Procedures)- Orchestration solutions using Integration Services

    • DQS Tecnologia

      Jan 2014 - Feb 2021
      Senior Data Engineer

      - Implement DataBricks solutions using Scala and Python.- Design and implementation of big data architectures- Scalable data designs with Data Lake- Mentor and help teammates do design solutions- Implement ETL/ELT Solutions for database integration within several companies- Implement several different kinds of POCs in order to adopt new tools- Design and implement Real-time streaming dashboards for a large oil & gas company - Development of APIs using Lambda, Python (Boto3) and DynamoDB- Leveraged (Spark) Databricks and Event Hub to build event driven data pipelines on Azure using Scala and Python.- Development/maintenance of several Data Products using AWS services like S3, Athena,Glue, Kinesis and Azure services like Azure Data Lake, Databricks (Spark), Azure Functions/EventHub/ EventGrid and Data Factory.- Logs files streaming ingestion from FluentD into S3 data lake using Kinesis Data Streams and Kinesis Firehose.- CDC using Kafka/Debezium and Spark Streaming- Data Extraction using Scrapy, Azure functions / AWS Lambda Show less

    • Elastacloud

      Feb 2021 - Jan 2023
      Senior Data Engineer

      • Implemented data pipelines using Databricks (Scala and Python) to process data from various security software APIs like Fireeye and CloudWatch.• Developed a streaming dashboard to detect suspicious login events efficiently.• Optimized data pipelines by migrating to Databricks, implementing advanced partitioning strategies, and optimizing join operations, reducing processing time by 60%.• Developed Azure Purview customizations

    • GSTi

      Jan 2023 - now
      Senior Data Engineer

  • Licenses & Certifications