AI Voice Evaluation Specialist in Haciendas del Canada, Nuevo León at Jobgether

NewJob Function: Human Resources
Jobgether
Haciendas del Canada, Nuevo León, 66054, Mexico
Posted on
New job! Apply early to increase your chances of getting hired.

Job Description

AI Voice Evaluation Specialist

This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for an AI Voice Evaluation Specialist based in Canada.

Join an innovative AI-focused environment as an AI Voice Evaluation Specialist, where you will contribute to improving the quality, reliability, and performance of next-generation conversational AI systems. In this role, you will interact directly with two different AI models through real-time voice conversations, using identical scenarios to assess and compare their performance. You will evaluate conversational quality, identify potential errors, and analyze how effectively each model responds to different situations. Your observations and structured assessments will help determine which systems deliver the strongest and most natural user experiences. This opportunity is ideal for someone with excellent listening skills, strong analytical abilities, and a keen interest in artificial intelligence and voice-based technologies. You will work independently within a structured evaluation framework, ensuring consistency, objectivity, and accuracy throughout each assessment.

Accountabilities:
  • Review assigned scenarios and conduct natural, voice-based roleplay conversations with two different AI models.
  • Interact with Model A and Model B using the same scenarios, maintaining consistent conversational approaches and conditions to ensure fair comparisons.
  • Keep approximately the same number of conversational turns with each model to support balanced and reliable evaluations.
  • Record voice interactions and carefully assess the quality, accuracy, relevance, and overall behavior of each model's responses.
  • Evaluate both AI models across five predefined quality dimensions, following established assessment criteria and evaluation guidelines.
  • Identify, document, and categorize relevant error patterns and clusters occurring during audio interactions or conversations.
  • Compare the performance of both models using structured evaluation frameworks and observations gathered throughout the interactions.
  • Determine which model demonstrates stronger overall performance based on the defined quality standards and evaluation criteria.
  • Prepare detailed written rationales supporting final evaluations, referencing specific conversational examples and observations from both models.
  • Apply evaluation guidelines consistently across different scenarios and interactions to ensure objective, accurate, and dependable results.
  • Maintain high standards of documentation, attention to detail, and evaluation consistency throughout all assigned tasks.
Requirements:
  • Bachelor's degree or equivalent relevant educational background.
  • Excellent attention to detail, with the ability to identify subtle differences in conversational quality, response accuracy, and AI model behavior.
  • Strong listening and comprehension skills, including the ability to understand spoken interactions and recognize inconsistencies or errors.
  • Excellent written communication skills, with the ability to document findings clearly, objectively, and accurately.
  • Ability to follow detailed instructions, structured evaluation frameworks, and predefined quality assessment guidelines.
  • Confidence in speaking naturally and participating in roleplay conversations across different scenarios.
  • Strong analytical and critical-thinking abilities, with the capacity to compare two interactions fairly and identify meaningful performance differences.
  • Objectivity and impartiality when evaluating AI-generated responses, without allowing personal preferences to influence final assessments.
  • High levels of reliability, consistency, and time management when completing repetitive and structured evaluation assignments.
  • Genuine interest in artificial intelligence, conversational AI assistants, voice technologies, or emerging AI applications.
  • Ability to work independently while maintaining accuracy and quality throughout the evaluation process.
Benefits:
  • Competitive hourly compensation ranging from CAD $20 to $25 per hour, depending on experience, skills, and qualifications.
  • Full-time contract opportunity for an initial period of 3 months.
  • 40-hour working week.
  • Remote working arrangement within Canada.
  • Opportunity to contribute directly to the evaluation and improvement of advanced AI voice technologies.
  • Hands-on exposure to conversational AI systems and emerging artificial intelligence applications.
  • Opportunity to develop practical experience in AI model evaluation, voice interaction analysis, and structured quality assessment.
  • Collaborative, technology-focused working environment with clearly defined evaluation processes and quality standards.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1

Explore Related Opportunities

Job Location

Haciendas del Canada, Nuevo León, 66054, Mexico

Frequently asked questions about this position

Apply For This Position

Apply Now