01 AI evaluation / Developer tooling
jevcheck
Does a model upgrade still honor the contract?
Behavioral contracts for TypeSafe Jev model upgrades. I built a Python library and CLI to pin expected behavior, record a baseline, compare candidate models, and fail CI when answers change or confidence regresses.
pip install jevcheckPyPI Listed in Hashgraph Online’s Awesome AI Plugins catalog.
| Case | Baseline | Candidate |
|---|---|---|
| 001 | billing0.93 confidence | billing0.93 confidenceUnchanged |
| 002 | general0.88 confidence | billing0.81 confidenceAnswer flip |
| 003 | billing0.92 confidence | billing0.71 confidenceConfidence drop |
Breaking · exit 1. One unchanged case, one answer flip, one confidence regression.