Providers & Models

AI Chat normalizes commercial, open and locally-hosted providers behind one model selector. Users move between fast inexpensive models, frontier reasoning models, private local models and specialized generation models without learning a different application for each.

Built-in providers​

The ChatApiKey static class names every environment variable AI Chat looks for:

ChatApiKey.OpenAI      // "OPENAI_API_KEY"
ChatApiKey.Anthropic   // "ANTHROPIC_API_KEY"
ChatApiKey.Gemini      // "GEMINI_API_KEY"

How a provider becomes live​

A provider is created and registered when all of these hold:

1. It has an entry in llms.json's providers object.
2. It is enabled - either "enabled": true in its definition, or named in ChatFeature.EnableProviders.
3. Its npm sdk id resolves to a factory in ChatFeature.ProviderTypes.
4. provider.Test() passes, which normally means an API key was resolved.

API keys are resolved in this order:

services.AddPlugin(new ChatFeature {
    Variables = {
        ["OPENAI_API_KEY"] = context.Configuration["OpenAi:ApiKey"]!,
        ["ANTHROPIC_API_KEY"] = context.Configuration["Anthropic:ApiKey"]!,
    },
});

Restricting the providers users can reach​

EnableProviders overrides every enabled flag in llms.json, which makes it the simplest way to pin a deployment to an approved set:

services.AddPlugin(new ChatFeature {
    // only these providers, whatever llms.json says
    EnableProviders = ["anthropic", "ollama"],
});

For a fully self-hosted deployment, enable only ollama or lmstudio - nothing then leaves your network.

llms.json​

App_Data/chat/llms.json is seeded from an embedded default on first run and is yours to edit:

{
  "version": 3,
  "disable_extensions": [],
  "defaults": {
    "headers": { "Content-Type": "application/json" },
    "text":  { "model": "DeepSeek V4 Flash", "messages": [] },
    "image": { "model": "MiMo-V2.5", "messages": [] }
  },
  "limits": {
    "client_timeout": 240,
    "client_max_size": 20971520,
    "retries": 3
  },
  "loading": ["Computing", "Cooking", "Crafting"],
  "providers": {
    "groq":      { "enabled": true },
    "anthropic": { "enabled": true },
    "ollama":    { "enabled": true }
  }
}

Supplying config in code​

Set Config (or ConfigJson) to bypass the App_Data file entirely - useful when configuration should come from your appsettings or a secret store:

services.AddPlugin(new ChatFeature {
    ConfigJson = File.ReadAllText("llms.json"),
});

Top-level $VAR values in Config are substituted on load, so "api_key": "$MY_KEY" works anywhere a key is expected.

providers.json and the model catalog​

App_Data/chat/providers.json holds the model catalog - ids, display names, context sizes, pricing and capability flags - sourced from models.dev. A provider definition in llms.json is merged over its providers.json entry, with the definition winning.

Refresh the catalog at runtime, keeping only your configured providers and layering in your own overrides:

var count = await feature.UpdateProviderModelsAsync();

providers-extra.json​

Models that aren't in models.dev - a private deployment, a preview model, a self-hosted checkpoint - go in App_Data/chat/providers-extra.json and survive every catalog refresh:

{
  "openai-local": {
    "models": {
      "my-finetune-v3": {
        "name": "My Finetune v3",
        "limit": { "context": 128000, "output": 8192 }
      }
    }
  }
}

Enabling and disabling at runtime​

Providers can be toggled without a restart. Both persist the change to llms.json and rebuild the live provider set:

// returns null on success, or the reason it couldn't be enabled
var error = await feature.EnableProviderAsync("anthropic");

await feature.DisableProviderAsync("openai");

var (enabled, disabled) = feature.ProviderStatus();

The Chat UI exposes the same operations, so an administrator can enable a provider from the browser once its key is present.

Local models​

Ollama and LM Studio need no API key - AI Chat just needs to reach their endpoint:

{
  "providers": {
    "ollama": {
      "enabled": true,
      "base_url": "http://localhost:11434/v1"
    }
  }
}

Any other OpenAI-compatible endpoint can be added under openai-local, or registered as a new sdk id - see Custom Extensions.

Modalities​

A provider definition can declare sub-providers for non-text modalities. Each is created as its own provider and attached to the parent:

{
  "providers": {
    "openai": {
      "enabled": true,
      "modalities": {
        "image": { "npm": "openai/image" }
      }
    }
  }
}

Built-in modality sdk ids include openai/image, openrouter/image, openrouter/audio, openrouter/text-to-speech, fireworks/image, zai/image, chutes/image, nvidia/image and mistral/transcriptions.

An unsupported modality sdk drops just that modality rather than disabling the whole provider. See Voice & Media.

Server tools​

Some providers host their own tools - web search, web fetch, code execution. AI Chat surfaces whatever the selected provider advertises, generating the configuration UI from the tool's JSON Schema:

Adding a provider type​

Register a new npm sdk id from an extension's Install:

public override void Install(ExtensionContext ctx)
{
    ctx.AddProvider("my-company/llm", () => new MyCompanyProvider());
}

Then reference it from llms.json:

{
  "providers": {
    "my-company": { "enabled": true, "npm": "my-company/llm", "api_key": "$MY_COMPANY_KEY" }
  }
}

Most services only need OpenAiCompatibleProvider with a different base_url.