Data Engineer | specialist in Brazil, Indiana at Jobgether
Explore Related Opportunities
Job Description
This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Data Engineer | Specialist in Brazil.
In this role, you will work at the intersection of Data Engineering and Artificial Intelligence, contributing to the modernization of legacy systems and data ecosystems through advanced, AI-driven solutions. You will design and implement scalable data pipelines, leveraging modern frameworks and cloud-native architectures to enable faster, more reliable data transformation and migration. The environment is highly technical and collaborative, bringing together data engineers, architects, and AI specialists to solve complex enterprise challenges. You will also play a key role in applying Generative AI to automate documentation, improve data lineage understanding, and accelerate legacy system analysis. This position offers strong exposure to Databricks and distributed data processing at scale. It is ideal for someone who thrives in innovation-driven environments and enjoys building reusable, impactful data solutions that shape enterprise-wide data strategy.
- Define and implement AI-driven solutions to support the modernization of legacy systems and data environments, enabling faster transformation and migration processes.
- Develop tools and mechanisms for extracting, analyzing, and interpreting technical information from legacy artifacts and systems.
- Build Generative AI solutions to automate documentation, transformation workflows, and data migration acceleration.
- Design intermediate metadata models representing business rules, dependencies, data flows, and transformations across systems.
- Develop reusable accelerators and components aligned with enterprise data architecture standards.
- Support the definition of templates, declarative frameworks, and best practices for modern data pipelines.
- Build and maintain scalable data pipelines using Databricks, PySpark, Lakeflow Jobs, and declarative pipeline frameworks.
- Work with batch processing, incremental ingestion, CDC (Change Data Capture), and enterprise system integrations.
- Contribute to data governance, lineage, traceability, and data quality improvements during migration initiatives.
- Collaborate closely with data architects, engineers, and platform specialists to design secure, scalable, and high-performance solutions.
This role requires strong technical expertise in Data Engineering combined with hands-on experience in Artificial Intelligence and Generative AI applied to complex data environments. The ideal candidate is highly proficient in building distributed data systems and comfortable working with modern AI frameworks and orchestration tools.
- Proven experience applying Artificial Intelligence in Data Engineering, Software Engineering, or technical automation contexts.
- Strong knowledge of Generative AI and Large Language Models (LLMs).
- Hands-on experience with AI agents, RAG architectures, prompt engineering, function calling, tool use, and automated AI workflows.
- Experience with orchestration frameworks such as LangChain, LangGraph, Semantic Kernel, CrewAI, or equivalent technologies.
- Advanced programming skills in Python.
- Strong experience with PySpark and Spark SQL in distributed data processing environments.
- Solid knowledge of SQL (advanced level) for data manipulation and querying.
- Experience building and maintaining scalable data pipelines in distributed architectures.
- Familiarity with data formats such as XML, JSON, and YAML.
- Knowledge of data modeling concepts and Data Warehouse architectures.
- Hands-on experience working in Databricks environments.
- Competitive compensation aligned with market standards
- Remote-first or hybrid work flexibility (depending on project allocation)
- Access to cutting-edge AI, GenAI, and data engineering technologies
- Continuous learning and upskilling in modern data and AI ecosystems
- Collaborative and innovation-driven environment with global exposure
- Career development opportunities in advanced data and AI projects
- Participation in high-impact enterprise transformation initiatives
- Health and wellness benefits (as per partner company policy).