OpenVals
4. Use the platform

OpenVals Integrations Analysis & User Documentation

Comprehensive analysis and user manual for connecting OpenVals to model registries, code repositories, databases, storage, and inference providers.

1. Integration Architecture & Current Maturity

The Integrations feature is connected to a dedicated backend service (Express + PostgreSQL, as described by the project types), separate from the main application API. Other dashboard areas—such as Model Compare and Command Center—still use the main API and/or mock-enabled data paths. Therefore, integrations are currently implemented at the integrations-service level and are not yet federated into all downstream features.

Integration levels

LevelMeaningCoverage
L1 — CatalogProviders are shown dynamically from GET /providers. Users can register custom connectors with POST /providers.All 50+ provider slugs listed in registry.ts.
L2 — Connect & storeA generic API-key dialog is driven by provider.config.form. Credentials are probed and stored through POST /credentials; connection state uses GET/DELETE /connections.Providers without a dedicated explorer.
L3 — Full explorerDedicated explorer UIs proxy live operations through GET/POST /api/integrations/{provider} with an action parameter. Provider credentials are sent in Authorization; the app JWT is sent in X-OpenVals-Token.Eight dedicated explorers plus a shared model-hub explorer for approximately 45 inference providers.

L3 explorer capabilities

Hugging Face
Identify the authenticated user (whoami); search models and datasets; inspect repository files and model details; run hosted inference; import models into the OpenVals catalog.
GitHub
Browse repositories, branches, file contents, commits, pull requests, issues, and Actions workflows/runs; dispatch workflows; push files; create branches, pull requests, and issues.
GitLab
Browse projects and groups, inspect project details, view pipelines, trigger pipelines, and configure self-hosted instance URLs.
Postgres
Connect and run queries; list/create databases and tables; preview, insert, update, and delete rows. Destructive operations require confirmation.
S3
Verify credentials; browse buckets and objects; inspect object metadata; generate presigned URLs; select a region.
Ollama
Ping a host; list, pull, inspect, and delete models; generate chat responses; create models from a Modelfile.
vLLM
Check service health; list served models; run completions; inspect metrics.
MLflow
Check the tracking URI; browse registered models, experiments, and runs; transition model stages; import models.
Shared model-hub explorer
For approximately 45 inference providers—including OpenAI, Anthropic, Gemini, Groq, Mistral, Together, and OpenRouter—connect, search, inspect model details, and run inference where supported. The chat tab is hidden when the provider configuration disables chat.

Important limitation: downstream model consumption

⚠️ Current gap

Models imported through POST /models currently appear in the Imported Models table on the Integrations page. Model Compare, Validation Runs, and Command Center still read from the main app API (which may use mock-enabled paths), so imported models are not yet consumed end-to-end by those features. This is an implementation gap, not merely a documentation issue.

2. User Guide

The Integrations page connects OpenVals to model sources, registries, repositories, databases, and storage. Connected credentials are verified when saved and stored encrypted by the integrations service.

Connecting a provider

  1. Open Integrations from the sidebar.
  2. Find the provider under Available Connectors and select Connect.
  3. Enter the requested credentials. Explorer providers (Hugging Face, GitHub, GitLab, Postgres, S3, Ollama, vLLM, and MLflow) open a dedicated explorer where the connection can be tested live. Other providers use a simple API-key dialog.
  4. After a successful connection, the provider card appears under Connected Integrations.

What you can do after connecting

  • Hugging Face and inference providers: Search and browse models and datasets, view model cards, run hosted inference where supported, and import models into the OpenVals catalog.
  • GitHub and GitLab: Browse repositories, branches, files, commits, pull requests/issues, and CI runs; create branches, pull requests, and issues; trigger pipelines or workflows where supported.
  • Ollama and vLLM: List locally served models, pull or create Ollama models, and send test prompts.
  • MLflow: Browse registered models, experiments, and runs; transition model stages; import models.
  • Postgres: Run queries and manage databases, tables, and rows. Destructive actions require confirmation.
  • S3: Browse buckets and objects and generate presigned download URLs.

Imported Models

Models imported from explorers appear in the Imported Models table on the Integrations page. The table includes fields such as name, provider, family, deployment, and lifecycle. Select Remove to delete an entry.

Custom connectors

Select Add Connector to register a custom provider in the catalog. Provide a slug, display name, type, icon, and optional connection-form labels. The connector then appears under Available Connectors.

Disconnecting a provider

On a connected provider card, select Disconnect and confirm. This deletes the stored credentials and closes the connection. You can reconnect later by following the connection steps.

Troubleshooting

“Integrations API unreachable”
The integrations service may be unavailable, or your session may have expired. Try Retry; if the issue persists, sign in again or contact your administrator.
Connection probe fails
The provider may have rejected the credential. Verify the API key/token, required scopes or permissions, provider endpoint, and any required region or instance URL.
Session expired
Sign in again when redirected. A failed provider credential probe should not, by itself, sign you out of OpenVals.