Lead Data Engineer at Jobgether – United States

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Lead Data Engineer in the United States.

This role is a high-impact opportunity to shape and scale a modern enterprise data platform built on Databricks, supporting advanced analytics and AI-driven initiatives. As a Lead Data Engineer, you will design and implement robust, scalable data pipelines that power critical business insights across the organization. You will work closely with architecture, analytics, and engineering teams to build efficient data transformation frameworks and ensure high-performance distributed processing at scale. The position combines hands-on technical development with leadership responsibilities, including mentoring engineers and defining best practices. Operating in a fast-evolving data environment, you will play a key role in enabling data products that support strategic decision-making. This is an ideal role for a senior engineer passionate about building high-quality, scalable data systems and driving engineering excellence.

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows on Databricks to support enterprise analytics and AI initiatives.
• Build and optimize data processing frameworks using PySpark, Spark, Python, and SQL for high-performance distributed computing.
• Develop robust data ingestion, transformation, and integration solutions across multiple data sources and systems.
• Establish engineering standards, best practices, and scalable architecture patterns while mentoring and guiding other data engineers.
• Collaborate with data architects, analysts, and business stakeholders to deliver reliable and high-quality data products.
• Monitor, troubleshoot, and optimize Spark workloads to ensure performance, scalability, and cost efficiency.
• Contribute to continuous improvement of data engineering processes, tools, and platform capabilities.

Requirements:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field.
• 5+ years of experience in data engineering, building large-scale data pipelines and distributed data systems.
• Strong expertise in Python and SQL for data processing and analytics.
• Extensive hands-on experience with Databricks, Apache Spark, and PySpark in production environments.
• Deep understanding of ETL/ELT processes, data modeling, and distributed computing concepts.
• Proven ability to mentor engineers and drive engineering best practices within a technical team.
• Strong communication and collaboration skills, with the ability to work across technical and business teams.
• Cloud experience with AWS or Azure and relevant certifications is a plus.
• Databricks certification is considered a strong advantage.

Benefits:

• Competitive salary ranging from approximately $150,000 to $175,000 annually, depending on experience and qualifications.
• Eligibility for annual discretionary bonus based on performance.
• Comprehensive benefits package including medical, dental, vision, disability, and life insurance.
• Retirement savings plan to support long-term financial security.
• Generous paid time off and work-life balance support.
• Inclusive and collaborative work environment focused on innovation and growth.
• Opportunities to work on modern data platforms and advanced analytics initiatives.
• Equal opportunity workplace committed to diversity, equity, and inclusion.