AI Data Scientist II in Rochester, New York at EagleView

Eagleview
Locations: Rochester, New York
Categories: Engineering
Job Type: Regular Full-Time
Work Type: Remote
Req ID: 2934
Job Description About Us

Eagleview, the leader in geospatial intelligence, is focused on building next-generation AI systems that turn high-resolution aerial imagery into dynamic, actionable insights. We are seeking an AI Data Scientist II to contribute to the evaluation and improvement of machine learning and agentic AI systems used in environmental monitoring, disaster response, urban planning, construction, government, and infrastructure management.

Overview

The AI Data Scientist II in AI, Data Science and Machine Learning will execute defined data science, experimentation, and model evaluation work for systems that interpret and use aerial imagery. Working with AI and data science, engineering, and product partners, this role will develop and apply metrics, test methods, analysis workflows, and evaluation tools that support product objectives. The role will solve technical problems limited in scope, make evidence-based judgments and proposals, and complete task-level deliverables with minimal coaching.

We are a fast-paced, collaborative team driven by continuous improvement. We are looking for a motivated and organized emerging professional who learns new technologies and development lifecycle practices, communicates effectively across team boundaries, and documents work so that others can understand and build on it.

This is a full-time, remote role, with a base salary of $98,000 - $134,000, bonus eligible.

Responsibilities

Model and Agentic System Evaluation

• Execute benchmarking tasks for machine learning and agentic AI systems using defined measures such as task success, accuracy, robustness, and common failure patterns.
• Apply and help maintain established evaluation methods for computer vision and machine learning models, including detection, segmentation, and classification models.
• Develop and maintain quantitative metrics, test harness components, and curated evaluation datasets for realistic, production-representative scenarios.
• Analyze model outputs, errors, and test results to identify performance issues and propose practical improvements.
• Conduct defined comparisons across model architectures, prompt strategies, tools, and inference configurations, then summarize results and recommendations.

Evaluation and Analysis

• Execute defined offline evaluations, including model comparisons, stress tests, and edge-case testing.
• Perform structured error analysis and root-cause investigation across data, model, and system failures, escalating broader issues when appropriate.
• Document code, data, configurations, and evaluation results so that work can be repeated, reviewed, and improved by others.
• Compare model accuracy, reliability, processing time, and resource use, and communicate findings clearly to stakeholders.

Cross-Functional Collaboration and Communication

• Work with AI engineers, product managers, and research partners to translate evaluation results into clear, actionable recommendations.
• Provide evidence-based findings and proposals that support model selection, system design, and product objectives.
• Participate in model reviews by preparing analysis, documenting limitations, and raising identified risks.
• Regularly communicate work status, raise issues with potential solutions, and document methods, results, and new concepts for technical and non-technical audiences within and beyond the team.

Tooling and Evaluation Support

• Develop and maintain repeatable scripts, notebooks, and test components for evaluating models and AI applications.
• Track agreed performance measures and identify regressions or unexpected results using established team standards.
• Support internal tools for evaluating models, visualizing metrics, and documenting test results.
• Work with engineering partners to incorporate evaluation results into development and testing workflows.
• Learn and apply current practices in agent evaluation, computer vision benchmarking, and machine learning performance measurement.
• Suggest improvements to metrics, benchmarks, and evaluation protocols as system capabilities and product needs evolve.
• Follow team practices for responsible AI, data quality, and appropriate documentation of model limitations.
• Other duties as assigned.

Qualifications

• Bachelor's degree or equivalent practical experience. Coursework, certification, or applied training in data science, machine learning, statistics, computer science, or a related area is preferred.

• Two to four years of relevant experience in data science, machine learning, model evaluation, or a related technical field.

• Hands-on experience evaluating machine learning models using appropriate quantitative metrics and clearly documenting results.

• Working knowledge of computer vision model evaluation, such as segmentation, detection, or classification metrics.

• Exposure to LLM-based, multimodal, or agentic systems and multi-step evaluation workflows.

• Proficiency in Python and commonly used machine learning and data science libraries, such as NumPy, Pandas, or scikit-learn.

• Experience using Git or another version control system to support collaborative technical work.

• Experience working in at least one cloud-based environment, such as AWS or GCP.

Preferred

• Experience evaluating image classification, document classification, LLM, or RAG applications.
• Experience creating reusable evaluation scripts, notebooks, dashboards, or quality assurance tools.
• Experience working in a collaborative software or machine learning development lifecycle.

Core Competencies

The successful candidate will demonstrate strength in the following competencies as well as foundational competencies which can be found here:

• Drive & Follow Through – Takes initiative and turns ideas into action.
• Adaptability in Uncertainty – Adjusts quickly and performs through change.
• Customer-Centered Mindset – Puts customer needs at the center of decisions.
• Judgment & Problem Solving – Makes sound decisions and solves problems effectively.
• Role-Specific Expertise – Applies strong functional expertise to deliver results.
• Results Accountability – Owns outcomes and delivers on commitments.
• Work Prioritization & Execution – Focuses on priorities and delivers on time.

EEO Statement

This job description is not an exclusive or exhaustive list of all job functions that a workforce member in this position may be asked to perform. Duties and responsibilities can be changed, expanded, reduced, or delegated by management to meet the business needs of the company.

The compensation offered to the successful candidate will be based on a variety of factors, including the candidate's work experience, education and licenses, work-related training, key skills, the core duties of the role and its associated responsibilities, additional benefits offered, and the location where the work will be performed. All Fulltime (30+ hours) employees are eligible for PTO, Sick, and Parental Leave; Medical, Dental, and Vision Insurance; 401(k) Plan; Health Savings Account; Life Insurance; Employee Assistance Program; Pet Insurance. This is a full-time, remote role, with a base salary of $98,000 - $134,000, bonus eligible.

As an Equal Opportunity and E-Verify Employer, Eagleview Technologies does not discriminate on the basis of any legally protected status or characteristic. Protected veterans and individuals with disabilities are encouraged to apply. We are committed to giving all applicants equal opportunity to participate in the application process and are open to discussing reasonable accommodations for candidates with disabilities.