Testing Lead-QA in India at Jobgether

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Testing Lead-QA based in India.

This is an exciting opportunity for an experienced quality engineering professional to lead testing initiatives for cutting-edge AI products, including Deep Learning, Large Language Models (LLMs), and Vision-Language Models (VLMs). The role combines technical leadership, test strategy development, and documentation ownership in a fast-paced and innovation-driven environment. You will work closely with machine learning engineers, product teams, and software developers to ensure AI systems are reliable, scalable, and production-ready. This position is ideal for someone who enjoys solving complex quality challenges in non-deterministic systems and building structured processes around emerging technologies. You will play a key role in shaping quality standards, improving AI evaluation methodologies, and fostering a quality-first culture across the organization.

• Define and lead end-to-end testing strategies for Deep Learning, LLM, and VLM products and pipelines.
• Establish testing frameworks covering model evaluation, acceptance criteria, release readiness, and risk assessment.
• Create and maintain comprehensive documentation related to testing methodologies, model assumptions, known limitations, and quality sign-offs.
• Design and execute testing strategies for prompt engineering, RAG pipelines, hallucination control, multi-turn conversations, and long-context model behavior.
• Develop and manage golden datasets, regression testing suites, and benchmarking processes.
• Evaluate multimodal and vision-language systems, including image-text alignment, OCR, captioning, and reasoning capabilities.
• Build Python-based automation frameworks for model evaluation, validation, and regression testing.
• Integrate testing processes into CI/CD and MLOps pipelines to support continuous delivery and model monitoring.
• Generate quality reports, dashboards, and actionable insights for engineering and leadership teams.
• Monitor production performance, identify model drift or degradation, and document behavioral changes across model versions.
• Establish scalable QA standards and mentor teams on testing best practices, documentation, and quality processes.
• Serve as the primary reference point for AI quality standards, testing governance, and risk management.

• 3–4 years of experience in software testing, with significant ownership or leadership experience in AI, Deep Learning, LLM, or Generative AI testing environments.
• Strong hands-on experience testing non-deterministic AI systems, machine learning models, and advanced language models.
• Excellent Python programming skills with experience in test automation, data validation, and quality engineering frameworks.
• Strong understanding of transformer architectures, deep learning workflows, and model evaluation methodologies.
• Proven ability to create clear, structured, and maintainable technical documentation.
• Experience developing testing strategies for prompt engineering, conversational AI systems, and retrieval-augmented generation (RAG) architectures.
• Familiarity with CI/CD pipelines, MLOps practices, and production monitoring for AI systems.
• Strong analytical and problem-solving skills with the ability to work effectively in fast-paced and ambiguous startup environments.
• Excellent communication and stakeholder management skills, with the ability to collaborate across technical and non-technical teams.
• Experience with Vision-Language Models, multimodal AI systems, or computer vision technologies is highly desirable.
• Familiarity with tools such as LangChain, LlamaIndex, MLflow, vector databases, and embedding technologies is considered a strong advantage.
• Exposure to AI governance, compliance requirements, and documentation of model risks and limitations is a plus.

• Flexible working options, including remote, hybrid, or on-site arrangements depending on business needs.
• Opportunity to work on cutting-edge AI technologies, including LLMs, Deep Learning, and multimodal systems.
• High-impact role with significant ownership and visibility across product and engineering teams.
• Collaborative startup environment that encourages innovation, experimentation, and rapid decision-making.
• Exposure to advanced AI architectures, MLOps practices, and next-generation quality engineering methodologies.
• Strong opportunities for professional growth, leadership development, and technical learning.
• Dynamic and cross-functional work environment with direct collaboration with machine learning, engineering, and product experts.
• Opportunity to shape quality standards and processes in an evolving and rapidly growing AI ecosystem.