Chat API

Registering ChatFeature gives your App an OpenAI-compatible Chat Completions endpoint and an in-process client, both running the same pipeline: provider selection, retry and failover, the tool-execution loop, usage and cost accounting, and every registered extension filter.

POST /v1/chat/completions​

The typed ChatCompletion service is mounted unprefixed at the standard OpenAI path, regardless of RoutePrefix:

curl https://your-app.example.com/v1/chat/completions \
  -H "Authorization: Bearer $MY_APP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Claude Sonnet 5",
    "messages": [
      { "role": "user", "content": "Capital of France?" }
    ]
  }'

Authentication is an Identity cookie or a Bearer API Key when ApiKeysFeature is registered. ValidateRequest runs before the completion, so quotas and policy apply here too.

Because the endpoint is OpenAI-compatible, any OpenAI SDK works against it by changing the base URL - including typed clients in 15 languages generated from your App's own metadata.

IChatClient​

For C# code inside the App, IChatClient runs the same pipeline without the HTTP round-trip:

public class SummaryServices(IChatClient chat) : Service
{
    public async Task<object> Any(SummarizeTicket request)
    {
        var ticket = await Db.SingleByIdAsync<Ticket>(request.Id);

        var response = await chat.ChatAsync(new ChatCompletion {
            Model = "Claude Sonnet 5",
            Messages = [
                new() {
                    Role = "user",
                    Content = [new AiTextContent {
                        Type = "text",
                        Text = $"Summarize this support ticket in two sentences:\n\n{ticket.Body}",
                    }],
                },
            ],
        });

        return new SummarizeTicketResponse {
            Summary = response.Choices[0].Message.Content,
        };
    }
}

public interface IChatClient
{
    Task<ChatResponse> ChatAsync(ChatCompletion request, CancellationToken token = default);
}

It throws on failure with the same exceptions the service surfaces, e.g. HttpError.NotFound when no configured provider serves the requested model.

Attributing usage to a user​

There's no IRequest to authenticate an in-process call, so attribute it explicitly with Metadata["user"]. Omitting it records the usage against the "default" user, matching how the feature behaves with RequireAuth = false:

var response = await chat.ChatAsync(new ChatCompletion {
    Model = "Claude Sonnet 5",
    Messages = [ /* ... */ ],
    Metadata = new() { ["user"] = base.GetSession().UserName },
});

This is what makes the call show up correctly in Analytics and be billed to the right user.

Chat UI routes​

The Chat UI's own endpoints are dispatched through ChatFeature's route registry under RoutePrefix:

Extension routes are namespaced under /chat/ext/{extension}/, e.g. /chat/ext/gemini/filestores. See each feature's page for its routes, and Custom Extensions for adding your own.

Working with the model catalog​

var models = feature.GetActiveModels();       // JsonArray across all live providers
var (enabled, disabled) = feature.ProviderStatus();

await feature.EnableProviderAsync("anthropic");
await feature.DisableProviderAsync("openai");
await feature.UpdateProviderModelsAsync();    // refresh from models.dev

The model catalog is also available to #Script as Chat.Models:

{‎{ Chat.Models | take(10) | join(', ') }‎}

Per-thread request args​

The Chat UI can set a whitelisted set of per-thread request arguments, which are forwarded to the provider:

ChatFeature.RequestArgs is the authoritative set; anything else supplied by the UI is ignored.

The pipeline in brief​

1. OnRequestAsync resolves any Bearer API key onto the request and runs first-request-per-user setup handlers.
2. ValidateRequest may reject the request.
3. Chat request filters run, letting extensions mutate the outgoing request.
4. A provider is selected for the requested model, with retry and failover per Limits.Retries.
5. Tool calls are executed in a loop, bounded by Limits.MaxIterations, pausing for approval where required.
6. Streamed responses are checkpointed every Limits.StreamCheckpointInterval into ChatThread.StreamingMessage, kept out of the durable Messages so a failed stream can't damage the conversation.
7. Chat response filters run, and usage and cost are recorded as a ChatRequest row.

Cancellation is cooperative - ShouldCancelThread consults the thread's state, so cancelling in the UI stops the loop.

Related​

• Providers & Models - what model names resolve to
• Tools - the registry the tool loop draws from
• Data & Storage - what a completion persists