Jev probability, confidence and human-review thresholds

No. In Jev, probabilities describe the distribution of a typed answer, while confidence summarizes its concentration. For a Choice with n labels and top probability p, TypeSafe documents confidence = (p - 1/n) / (1 - 1/n). An illustrative three-label answer with probabilities 0.90, 0.06 and 0.04 has confidence 0.85. Confidence is not an independently measured accuracy rate. JevModel passes through the upstream answer; validate actual fields before use.

Noul returns the probability of yes in the noul field and has no separate upstream confidence field. A value near 0.5 suggests uncertainty about that binary question. Score evaluates ordered levels and can return a fractional value; its numeric answer is not a probability. Interpret a Score against its rubric, and inspect the returned distribution and confidence where available.

Send ambiguous inputs, unsupported topics, malformed responses and service errors to a review queue. Add an explicit review label where appropriate, but do not rely on the model to always choose it. Gate automated routes using validated labels and a threshold tested for that question. Close leading probabilities can signal ambiguity. Keep deterministic permissions and human confirmation for sensitive actions even when the model is confident.

Separate instruction tuning, threshold selection and a final held-out test. On labeled inputs, report errors among automatically accepted cases together with coverage: how many cases remain automated after review. Check each class and rare high-cost mistakes. Reliability diagrams and Brier scores can assess probability calibration; evaluate probabilities rather than treating confidence as a correctness probability. Recheck after changing labels, instructions or upstream model behavior. No universal threshold is guaranteed here.