Supported Benchmark Domains
OpenVals offers a variety of specialized benchmark datasets tailored to evaluate models across different industries, reasoning tasks, and technical domains.
Available Datasets
| Dataset ID | Domain | Description |
|---|---|---|
| finance | finance | Financial reasoning and risk evaluation |
| healthcare | healthcare | Clinical reasoning and safety validation |
| legal | legal | Legal reasoning and compliance validation |
| developer | developer | Coding and software engineering evaluation |
| sample | general | Basic sample evaluation dataset |
| cybersecurity | cybersecurity | Cybersecurity reasoning and threat analysis |
| reasoning | reasoning | General reasoning and problem-solving evaluation |
| math | math | Mathematical reasoning and problem-solving evaluation |
| enterprise_ops | enterprise_ops | Enterprise operations and decision-making evaluation |
Finance (finance)
Financial reasoning and risk evaluation. Tests model capabilities in understanding financial statements, computing ratios, and analyzing compliance and market risk factors.
Healthcare (healthcare)
Clinical reasoning and safety validation. Evaluates clinical question answering, medical summary accuracy, and alignment with healthcare safety guidelines.
Legal (legal)
Legal reasoning and compliance validation. Benchmarks models on legal text analysis, contract review intelligence, regulatory interpretation, and compliance checks.
Developer (developer)
Coding and software engineering evaluation. Assesses code generation, debugging accuracy, system design reasoning, and algorithm optimization skills.
Sample (general)
Basic sample evaluation dataset. A lightweight dataset designed for quick smoke tests, initial pipeline verification, and local execution configuration.
Cybersecurity (cybersecurity)
Cybersecurity reasoning and threat analysis. Assesses models on vulnerability identification, threat modeling, security log interpretation, and incident response logic.
Reasoning (reasoning)
General reasoning and problem-solving evaluation. Tests logical inference, causal reasoning, multi-step deduction, and abstract problem-solving benchmarks.
Math (math)
Mathematical reasoning and problem-solving evaluation. Challenges models with algebra, calculus, discrete math, and word problems requiring symbolic reasoning.
Enterprise Operations (enterprise_ops)
Enterprise operations and decision-making evaluation. Assesses operational planning, resource allocation modeling, business logic analysis, and process optimization.