Engenheiro de Dados Sênior| IA Generativa, Dados e MDM in Brazil, Indiana at Jobgether

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Engenheiro de Dados Sênior | IA Generativa, Dados e MDM based in Brazil.

This role offers the opportunity to design and evolve advanced data solutions powered by Artificial Intelligence, Generative AI, and modern data engineering practices.
You will work on strategic initiatives involving customer data platforms, intelligent agents, and large-scale data processing environments.
The position combines data engineering, machine learning, governance, and automation to create solutions that generate measurable business impact.
You will contribute to building scalable architectures, improving data quality, and enabling smarter decision-making through trusted information.
The environment encourages collaboration between technology, data, and business teams while promoting innovation and continuous improvement.
You will have the opportunity to work with cutting-edge technologies such as Databricks, Spark, LLMs, and intelligent workflows.

As a Senior Data Engineer, you will be responsible for designing, developing, and optimizing scalable data solutions that support AI-driven initiatives, data governance, and business intelligence. Your responsibilities will include:

• Developing, optimizing, and maintaining scalable data pipelines using Databricks, Spark, and PySpark;
• Integrating, transforming, processing, and making customer data available across large-scale environments;
• Building and maintaining cloud-based Big Data products, ensuring scalability, performance, reliability, and data quality;
• Supporting initiatives related to customer data management, Golden Record, data quality, governance, and MDM;
• Designing data solutions that enable Generative AI applications, autonomous agents, multi-agent systems, and intelligent workflows;
• Preparing and structuring data for consumption by Machine Learning models, LLMs, AI agents, and automated processes;
• Supporting the development of intelligent agents capable of analyzing large volumes of customer data;
• Identifying patterns, inconsistencies, duplicates, anomalies, and opportunities for improvement within data assets;
• Developing mechanisms for alerts, explainable recommendations, and data-driven decision support;
• Applying Analytics and Machine Learning techniques for anomaly detection, classification, clustering, and scoring;
• Supporting data enrichment, prioritization, and governance strategies;
• Exploring and implementing architectures such as RAG, autonomous agents, and intelligent workflows;
• Working with near real-time data processing solutions using Kafka;
• Supporting Machine Learning model production in Databricks through MLOps practices;
• Creating and maintaining CI/CD pipelines using GitHub and GitHub Actions;
• Ensuring engineering best practices related to quality, efficiency, maintainability, testing, and governance;
• Collaborating with Data, Technology, MDM, and Business teams to deliver measurable and applicable solutions;
• Participating in technical refinements, process analysis, and AS-IS/TO-BE documentation;
• Identifying technical debt and proposing continuous improvements in architecture, processes, and data solutions.

We are looking for a senior professional with strong experience in data engineering, cloud environments, and AI-driven data solutions. The ideal candidate should have:

• Solid experience as a Data Engineer, Senior Data Engineer, or similar role;
• Strong knowledge of Python for data engineering, automation, analytics, and AI solution development;
• Advanced SQL skills and experience working with structured data exploration;
• Hands-on experience with Databricks, PySpark, and Apache Spark;
• Proven experience building, optimizing, and maintaining scalable data pipelines;
• Experience working with cloud environments, preferably Azure Databricks;
• Knowledge or experience with Gemini and Generative AI solutions;
• Experience with LLMs, AI-powered applications, and intelligent data workflows;
• Familiarity with RAG architectures, recommendation systems, cognitive automation, and intelligent workflows;
• Experience applying Machine Learning techniques such as anomaly detection, classification, clustering, and scoring;
• Experience with near real-time data processing, preferably using Kafka;
• Experience deploying Machine Learning models in Databricks and applying MLOps practices;
• Experience with CI/CD pipelines, especially GitHub and GitHub Actions;
• Knowledge of data engineering best practices, version control, testing, code review, and data quality processes.

Additional skills that are considered a plus:

• Experience working with critical data domains such as customer identity, registration, or KYC;
• Knowledge of data quality, governance, MDM, and Golden Record concepts;
• Experience with multi-agent systems, cognitive automation, or autonomous decision-making solutions;
• Experience working in regulated environments;
• Knowledge of deduplication, matching, consolidation, and customer data enrichment strategies;
• Experience using AI-assisted development tools, prompt engineering, or automated code review solutions.

• Health and dental insurance;
• Meal and food allowance;
• Childcare assistance;
• Extended parental leave;
• Access to fitness and wellness partnerships through Wellhub and TotalPass;
• Profit-sharing program (PLR);
• Life insurance;
• Continuous learning platform and professional development opportunities;
• Discount club with partner benefits;
• Free online platform focused on physical, mental, and emotional well-being;
• Parenting and maternity support programs;
• Partnerships with online learning platforms;
• Language learning platform;
• Additional benefits according to company policies.