Can I apply directly for this job on this page?

Yes, you can begin your application on this page using a quick form. You'll then be redirected to the employer's career site to complete the full application process.

What is the role of a Senior AI Systems Quality Engineer at Jobgether?

The Senior AI Systems Quality Engineer position at Jobgether is a Full-time or part-time position opportunity in the Engineering field.

Where is this Senior AI Systems Quality Engineer job located?

United States, Other / Non-US, United States

What type of employment is offered for this Senior AI Systems Quality Engineer role?

Full-time or part-time position

What is the expected salary for this Senior AI Systems Quality Engineer job?

Compensation will be discussed during the hiring process.

Senior AI Systems Quality Engineer job near me in United States, Other / Non-US at Jobgether

Senior AI Systems Quality Engineer

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Senior AI Systems Quality Engineer in United States.

This role sits at the intersection of AI engineering, platform quality, and production reliability for advanced agentic systems used in mission-critical healthcare environments. You will help define how quality is embedded directly into AI systems rather than validated after deployment. The focus is on building scalable, automated testing and evaluation frameworks for LLM-driven and agent-based architectures. You will work closely with AI Engineers, platform teams, and product stakeholders to ensure safe, predictable, and auditable system behavior. The environment is highly technical, collaborative, and innovation-driven, with a strong emphasis on automation-first quality engineering. Your work will directly influence the trustworthiness, performance, and release readiness of AI systems operating at scale.

Accountabilities:

Design and build production-grade automated testing frameworks, evaluation pipelines, and validation systems across the full AI lifecycle, ensuring continuous quality enforcement from development through deployment.
Architect and maintain an AI testing platform integrated with tools such as Databricks and MLflow to support traceability, lineage, and auditability at scale.
Develop large-scale, scenario-based test suites to validate agentic workflows, including edge cases, long-tail behaviors, orchestration logic, and failure modes.
Define and operationalize quality signals for LLM and AI systems (e.g., hallucination rate, grounding, latency, cost, and relevance) and embed them into CI/CD pipelines as automated quality gates.
Validate non-deterministic system behavior and ensure safe degradation patterns under uncertainty, load, and system failure conditions.
Partner with AI, platform, security, and delivery teams to define system quality standards, risk thresholds, and release readiness criteria.
Enable continuous AI validation by ensuring automated testing is triggered on model, prompt, and code changes across deployment pipelines.

Requirements:

7+ years of software engineering experience, ideally in backend or platform engineering environments.
Strong experience designing and implementing automated testing frameworks for complex or distributed systems, particularly AI or data-intensive platforms.
Proficiency in Python and/or TypeScript, with experience in modern AI engineering stacks.
Hands-on experience with LLM-based systems, agentic workflows, or non-deterministic AI behaviors.
Deep understanding of CI/CD systems and how to integrate automated quality gates into deployment pipelines.
Experience building scalable evaluation, regression, or validation frameworks for AI systems at production scale.
Familiarity with cloud-native environments, particularly Amazon Web Services.
Strong understanding of AI system risks, including security, privacy, governance, and operational failure modes.
Ability to define measurable AI quality thresholds (e.g., accuracy, hallucination limits, bias, and explainability) and translate them into enforceable release criteria.
Strong collaboration and communication skills, with experience working alongside technical and non-technical stakeholders.
Nice to have: experience with Databricks ecosystems, MLflow-based workflows, observability tools, and LLM evaluation/guardrail frameworks.

Benefits:

Competitive compensation including base salary, performance-based bonuses, and equity eligibility.
Unlimited paid time off to support work-life balance and personal well-being.
Remote-first flexibility allowing you to work from anywhere in the United States.
Comprehensive healthcare coverage including medical, dental, vision, disability, and mental health support.
Equity participation, giving employees shared ownership in company success.
Home office setup allowance and monthly communication stipend.
Growth-oriented environment focused on continuous learning, innovation, and career development.

How Jobgether works:

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1

Senior AI Systems Quality Engineer in United States at Jobgether

Explore Related Opportunities

Job Description

Scan to Apply

Job Location

Frequently asked questions about this position