OpenVals Integrations Analysis & User Documentation
1. Integration Architecture & Current Maturity
The Integrations feature is connected to a dedicated backend service (Express + PostgreSQL, as described by the project types), separate from the main application API. Other dashboard areas—such as Model Compare and Command Center—still use the main API and/or mock-enabled data paths. Therefore, integrations are currently implemented at the integrations-service level and are not yet federated into all downstream features.
Integration levels
| Level | Meaning | Coverage |
|---|---|---|
| L1 — Catalog | Providers are shown dynamically from GET /providers. Users can register custom connectors with POST /providers. | All 50+ provider slugs listed in registry.ts. |
| L2 — Connect & store | A generic API-key dialog is driven by provider.config.form. Credentials are probed and stored through POST /credentials; connection state uses GET/DELETE /connections. | Providers without a dedicated explorer. |
| L3 — Full explorer | Dedicated explorer UIs proxy live operations through GET/POST /api/integrations/{provider} with an action parameter. Provider credentials are sent in Authorization; the app JWT is sent in X-OpenVals-Token. | Eight dedicated explorers plus a shared model-hub explorer for approximately 45 inference providers. |
L3 explorer capabilities
Important limitation: downstream model consumption
Models imported through POST /models currently appear in the Imported Models table on the Integrations page. Model Compare, Validation Runs, and Command Center still read from the main app API (which may use mock-enabled paths), so imported models are not yet consumed end-to-end by those features. This is an implementation gap, not merely a documentation issue.
2. User Guide
The Integrations page connects OpenVals to model sources, registries, repositories, databases, and storage. Connected credentials are verified when saved and stored encrypted by the integrations service.
Connecting a provider
- Open Integrations from the sidebar.
- Find the provider under Available Connectors and select Connect.
- Enter the requested credentials. Explorer providers (Hugging Face, GitHub, GitLab, Postgres, S3, Ollama, vLLM, and MLflow) open a dedicated explorer where the connection can be tested live. Other providers use a simple API-key dialog.
- After a successful connection, the provider card appears under Connected Integrations.
What you can do after connecting
- Hugging Face and inference providers: Search and browse models and datasets, view model cards, run hosted inference where supported, and import models into the OpenVals catalog.
- GitHub and GitLab: Browse repositories, branches, files, commits, pull requests/issues, and CI runs; create branches, pull requests, and issues; trigger pipelines or workflows where supported.
- Ollama and vLLM: List locally served models, pull or create Ollama models, and send test prompts.
- MLflow: Browse registered models, experiments, and runs; transition model stages; import models.
- Postgres: Run queries and manage databases, tables, and rows. Destructive actions require confirmation.
- S3: Browse buckets and objects and generate presigned download URLs.
Imported Models
Models imported from explorers appear in the Imported Models table on the Integrations page. The table includes fields such as name, provider, family, deployment, and lifecycle. Select Remove to delete an entry.
Custom connectors
Select Add Connector to register a custom provider in the catalog. Provide a slug, display name, type, icon, and optional connection-form labels. The connector then appears under Available Connectors.
Disconnecting a provider
On a connected provider card, select Disconnect and confirm. This deletes the stored credentials and closes the connection. You can reconnect later by following the connection steps.