Core Capabilities
Model Evaluation
Metrics
Evaluate AI systems using multiple dimensions including accuracy, semantic similarity, reliability, safety, consistency, variance, latency, factuality, and hallucination risk.
Evaluation Signals
| Metric | Ideal Direction | Good Range | Meaning |
|---|---|---|---|
| Accuracy | Higher | 0.80 to 1.00 | Correctness of output |
| Semantic | Higher | 0.75 to 1.00 | Meaning similarity and contextual alignment |
| Reliability | Higher | 0.70 to 1.00 | Stability across repeated generations |
| Safety | Higher | 0.85 to 1.00 | Lower risk and harmful behavior |
| Consistency | Higher | 0.75 to 1.00 | Repeatability of model behavior |
| Variance | Lower | 0.00 to 0.25 | Output deviation across runs |
| Latency | Lower | 0ms to 1500ms | Response generation speed |
| Factuality | Higher | 0.80 to 1.00 | Semantic factual alignment and lack of contradictions |
| Hallucination Risk | Lower | 0.00 to 0.20 | Estimated probability of hallucinated or unreliable content |
| DRS Score | Higher | 0.75 to 1.00 | Overall deployment reliability |