Model Sensitivity Evaluation Suite

Built at JWX to answer a concrete question: how much does prompt spec-detail actually matter once you factor in which model is running it? The harness cross-tests spec-detail level against model choice across Sonnet 5, Opus 5, and Fable 5, running ~90 evaluation runs spanning 3 models and 5 real engineering tickets.

Each run is scored on output quality against the ticket's actual requirements, not just pass/fail, so the results show which spec-detail level pays off per model rather than assuming one prompting style generalizes across providers. A live UI dashboard surfaces per-job cost and alarms as a supporting view into the run data.