AI Operations Engineer in Draper, Utah at BlytzPay

Location: Draper, UT (On-site)

Level: Entry to Mid-Level (Perfect for a high-performing recent CS/Data Science grad)

At BlytzPay, we are evolving the landscape of intelligent payments and collections through an AI-powered platform that connects payments, customer engagement, and automation into a single, frictionless layer.

We are looking for a Technical Bridge to take this platform from architecture to production-grade utility.

You will be the primary driver of Agent Skill Development. You will sit between our internal users and our core architecture, ensuring the AI is learning, executing tools correctly, and providing accurate value to the company.

or related field

Role

• Skill Construction: Use agentic coding to build agentic skills (third-party connections and tool-use logic).
• Troubleshooting: When our internal users are unable to train the agent out of a mistake via feedback, you step in. You’ll dig into the code, refine the prompts, and fix the hallucination or logic error.
• Human-in-the-Loop Management: Work directly with internal employees to manage evals. You’ll help them understand how to test the agent and translate their qualitative feedback into quantitative technical improvements.
• Architectural Strategy: You will work within the existing agentic architecture, providing feedback on how the system handles memory, context, and tool-triggering.

Requirements

• Agentic AI Expertise: Deep understanding of LLM memory management, context windows, and tool interaction/orchestration.
• Familiarity with Data Science Principles: Knowledge of data sanitization, evaluation metrics, and data pipelines.
• Exceptional Communication and Interpersonal Skills: Ability to translate complex technical concepts for non-technical stakeholders and act as a liaison to internal staff.
• Bachelors Degree in CS/Data Science