AI Quality Engineer in Toronto, Ontario at Q4 Inc

At Q4, we make an impact together, obsess over our customers, operate with integrity, and bring big ideas to life.

Q4 is charting a bold new path for investor relations as the first AI-driven IR Ops Platform, providing everything an IR team needs to succeed on a single, powerful platform. The Q4 Platform enables public companies to attract, manage, and understand investors - all in one place. Over 2,600 customers, including many of the most respected brands in the world, trust Q4 to help drive premium valuations for their companies. Only Q4 offers a tech stack holistically designed to equip IR teams with data, insights, and smart workflows that power remarkable outcomes. Learn more at q4inc.com.

We hire smart, curious, and talented people to push boundaries, reimagine what’s possible, and turn challenges into opportunities. All while keeping the needs of our clients at the heart of everything we do.

Come grow with us!

Position Title: AI Quality Engineer

Location: Canada

Main Job Role

You know the product deeply, think in risk, and use AI to amplify your impact. At Q4, this role is about quality strategy, planning, orchestration, AI system validation, and production intelligence. You bring agentic skills that extend your reach beyond traditional QE and help the team make better, faster quality decisions on the Q4 platform.

You will be working alongside the product squad, shaping quality from sprint planning through production monitoring. A core focus of this position is dedicating 60% to 70% of your time to utilizing AI to build automation, tooling, and reporting. The hard question you will help answer is: how do we design quality into AI-driven, probabilistic systems and measure it honestly?

Responsibilities

Quality Planning & Domain Mastery

Develop deep product and domain knowledge to design quality strategies that reflect real business risk, not just specification coverage.

Lead quality planning during sprint ceremonies by defining acceptance conditions, surfacing risk early, and translating requirements into testable outcomes.

Engage in design and architecture discussions to identify testability gaps, edge cases, observability needs, and failure modes before implementation begins.

Own the squad-level coverage model and risk-based test strategy for our industry-leading financial intelligence platform.

AI Augmented Test Orchestration

Use AI-powered workflows to accelerate test planning, exploratory analysis, scenario generation, and risk prioritization.

Design LLM-as-judge patterns, eval harnesses, and exploratory agents that help scale quality thinking beyond scripted test suites.

Partner with engineering to define practical quality gates in CI/CD, including coverage expectations, release signals, and regression confidence.

Drive test observability through coverage trends, flake rates, regression velocity, release readiness, and quality signal health.

AI & Eval System Quality

Deploy and test cutting-edge AI solutions directly on top of the Q4 platform.

Own the test strategy for LLM-backed and agentic product features, including prompts, chains, tool use, multi-step reasoning flows, and RAG pipelines.

Design and implement AI evaluation frameworks using tools such as RAGAS, DeepEval, LangSmith, or custom LLM-as-judge approaches.

Calibrate evaluations to the product context instead of relying only on generic, off-the-shelf scoring.

Quality Intelligence & Production Health

Use production signals to identify quality risks, diagnose failures, and improve release confidence.

Lead technical RCAs with AI-assisted analysis by tracing failures across services, logs, prompts, tools, and user workflows.

Translate production findings into prioritized backlog items with clear impact framing for Product and Engineering.

Requirements

Required Skills & Experience

A strong product-specific mindset with a holistic approach to quality, prioritizing software reliability and automation.

Demonstrated technical depth in quality engineering; flexible seniority allowing for intermediate or highly capable junior candidates with the right AI skill set.

Hands-on proficiency with AI evaluation testing, specifically utilizing eval harnesses and LLM-as-a-judge methodologies.

Technical familiarity with our core stack: Playwright for automation, Claude for AI functionality, and the MERN stack (MongoDB, Express, React, and Node) for the platform.

Proven AI fluency in practice, having used LLMs, agents, or AI-assisted workflows to improve the quality, speed, or depth of your work.

Agile/Scrum fluency and experience with tools such as Jira, TestRail, or equivalent systems.

Strong Assets

Experience building AI-assisted or autonomous test orchestration workflows.

Performance and reliability testing experience using tools such as k6, Locust, or similar.

SaaS B2B product experience, especially in fintech, investor relations, financial intelligence, or regulated domains.

This role is not a fit for:

Someone who sees quality primarily as manual test execution.

Someone who waits for fully defined requirements before engaging in quality work.

Someone who has not yet started using AI tools in their own quality or engineering practice.

Compensation & Benefits

Salary Range: $90,000 – $135,000 CAD per year.

Placement: The starting target range begins around $90,000 CAD to accommodate intermediate or highly capable talent engineering their way into AI quality, with budget flexibility up to $135,000 CAD for exceptional expertise.

Interview Process

Stage 1: A one-hour cultural fit interview which will include a short, real-time assessment to demonstrate practical AI tool usage.

Stage 2: A one-hour technical interview with the engineering team.

Why This Role

Quality at Q4 is not a checklist. It is a signal system that tells us whether an AI-driven financial intelligence platform is trustworthy.

You will help architect that system for your product: measuring what matters, diagnosing what breaks, validating probabilistic behavior honestly, and making AI work for quality rather than around it.