Data Engineer, Sr. in Annan Water, Scotland at Jobgether

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineer, Sr. based in United Kingdom.

Join a high-impact engineering team focused on modernizing large-scale data infrastructure and supporting critical database migration initiatives. In this role, you will design, build, and optimize cloud-based data pipelines while ensuring the reliable movement and transformation of structured and unstructured data. You'll work with modern AWS technologies, data lakes, and warehouse architectures to deliver scalable, high-performance solutions. This position offers the opportunity to solve complex technical challenges, collaborate with cross-functional teams, and contribute to product-driven development in a fully remote environment. If you enjoy building resilient data platforms and driving cloud innovation, this is an excellent opportunity to make a meaningful impact.

• Design, develop, and maintain scalable ETL pipelines to support large-scale data migration and integration projects.
• Build, configure, and optimize AWS-based data lake and warehouse solutions using services such as Glue, S3, Athena, Redshift, Lake Formation, Lambda, EventBridge, and Step Functions.
• Develop Python scripts and database procedures to automate data processing, transformation, validation, and migration workflows.
• Manage relational databases, including MySQL, SQL Server, and Aurora MySQL, with a focus on performance, replication, and reliability.
• Configure and troubleshoot AWS data engineering services, including Glue Catalog, Hudi tables, SQS, and CloudFormation resources.
• Monitor data platforms, identify performance bottlenecks, and implement improvements to ensure scalability, security, and operational excellence.
• Collaborate with product, engineering, and business teams while contributing to Agile development processes and technical planning.

• Proven experience as a Data Engineer with strong expertise in AWS cloud data engineering and migration projects.
• Hands-on experience with AWS services including S3, Athena, Glue, Redshift, Lake Formation, Lambda, Step Functions, EventBridge, CloudFormation, and RDS.
• Strong knowledge of relational databases, including MySQL, SQL Server, and data warehouse architecture, particularly Amazon Redshift.
• Proficiency in Python for scripting, automation, ETL development, and data transformation.
• Experience working with structured and unstructured data, data cleansing, conversion, and large-scale ETL processes.
• Knowledge of Hudi table management, AWS Glue Catalog, SQS, and cloud-based data lake implementations.
• Familiarity with Agile methodologies and product-focused software development environments.
• Excellent analytical, troubleshooting, communication, and leadership skills with the ability to work independently in a remote setting.
• Experience with AWS Database Migration Service (DMS), Scala, or Aurora MySQL replication is considered an advantage.

• Fully remote work environment with flexibility to work from anywhere.
• Competitive contract compensation based on experience.
• Opportunity to work on large-scale cloud migration and modern data platform initiatives.
• Exposure to a broad AWS ecosystem and advanced cloud-native technologies.
• Collaborative Agile environment with high levels of ownership and technical autonomy.
• Opportunity to contribute to innovative, product-driven engineering projects and expand cloud data engineering expertise.