Senior Machine Learning / Data Engineer - Evaluation

About VTI Aerospace VTI Aerospace builds AI\-powered perception and pilot assist technologies to unlock the future of aviation. With offices in Bozeman, MT and Seattle, WA, we are a team of engineers and technologists from Boeing, Airbus, Aurora, and beyond. We are passionate about pushing the boundaries of autonomy and aviation safety. Our company is dedicated to advancing the field of aviation with cutting\-edge solutions The Role As a Senior Software Engineer – Evaluation, you will design and implement systems that measure and monitor the performance of our computer vision, automatic speech recognition (ASR), and small language model (SLM) systems. You will develop evaluation methodologies, benchmarking pipelines, and monitoring tools that ensure our AI systems perform reliably in real\-world environments. You will help establish the evaluation standards and performance benchmarks that guide the development of all AI systems at the company. You will work closely with machine learning and data engineering teams to evaluate model performance, identify failure modes, and guide improvements to data collection and model training, helping bring AI\-powered aviation tools to market. This role is ideal for someone who enjoys designing robust evaluation systems and uncovering insights from model performance data, and wants to have a direct impact on the quality and reliability of our AI systems. What You'll Do * Define and implement evaluation methodologies for computer vision, ASR, and language systems * Identify and track key performance indicators (KPIs) that measure system and model effectiveness * Perform dataset coverage analysis to understand strengths, gaps, and biases in training and evaluation data * Identify model deficiencies and collaborate with ML engineers to improve training data and model performance * Build scalable model evaluation pipelines in Python for automated benchmarking and regression testing * Design and maintain syste