Core Capabilities
Model Evaluation
Metrics
Evaluate model outputs against structured datasets using accuracy, embedding-based semantic similarity, reliability, safety, consistency, variance, and latency.
Evaluation Signals
| Metric | Ideal Direction | Good Range | Meaning |
|---|---|---|---|
| Accuracy | Higher | 0.80 to 1.00 | Correctness of output |
| Semantic | Higher | 0.75 to 1.00 | Meaning similarity and contextual alignment |
| Reliability | Higher | 0.70 to 1.00 | Stability across repeated generations |
| Safety | Higher | 0.85 to 1.00 | Lower risk and harmful behavior |
| Consistency | Higher | 0.75 to 1.00 | Repeatability of model behavior |
| Variance | Lower | 0.00 to 0.25 | Output deviation across runs |
| Latency | Lower | 0ms to 1500ms | Response generation speed |
| DRS Score | Higher | 0.75 to 1.00 | Overall deployment reliability |