Data Engineer at OmegaHires – Charlotte, North Carolina

Job Title: Data Engineer
Job Type: Contract
Duration: Long-term
Location: Charlotte, NC

Description:

We are looking for a Data Engineer to join our team and help build out our Data Mesh platform, focusing on Ingestion as a Service. The ideal candidate will work on Spark-based frameworks, leverage AWS native services, and apply functional programming principles to deliver scalable, cloud-native data solutions. This role will collaborate closely with both business and technology teams to deliver robust, consumer-focused data platforms.

Key Responsibilities:

• Design and implement AWS cloud-native solutions for Data Mesh, specifically the Ingestion as a Service module.
• Build and enhance Spark-based frameworks to support scalable and reusable data ingestion pipelines.
• Apply functional programming concepts to develop modular, maintainable microservices for the data mesh domain.
• Develop, build, and test data engineering pipelines using Python and PySpark.
• Collaborate with business and technology stakeholders to deliver modern, consumer-focused data platforms.
• Contribute to the development of technology standards, tools, and processes in support of the overall data strategy.
• Assist with conceptual data modeling and ensure alignment with data domain principles.
• Communicate and apply data management concepts such as Master Data Management, Critical Data Elements, Granularity, and Normalization.
• Operate and manage CI/CD pipelines using Jenkins, CloudFormation, etc.
• Work with AWS Data Services including Glue, Lambda, Step Functions, DynamoDB, RDS (Postgres), Event Triggers, and Lake Formation.