chat-backend

@evolve-framework/chat-backend · v2.2.0

Runs the backend of an AI shopping assistant, as a module that serves the chat API for the Evolve chat widget and answers with tools from an MCP server or a GraphQL API.

When to use

Use it when a shop offers an AI shopping assistant through the Evolve chat widget. One ChatBackendConfig describes one assistant: its model, system prompt, store context, auth cookies and tools.

• MCP: set config.mcp, and the assistant uses the tools of that MCP server. The shopper's auth cookies are forwarded as headers.
• GraphQL: set config.graphql, and the assistant gets a single query tool that runs the GraphQL queries it writes. schemaPath puts the schema in the system prompt.
• Models: OpenAI, Google, Azure OpenAI and Azure AI Foundry, set per deployment in config.model with its own API key.
• Several assistants in one service: mount a ChatBackendModule per assistant, each with its own basePath and config.id.

Set REDIS_URL, or UPSTASH_REDIS_REST_URL with UPSTASH_REDIS_REST_TOKEN, to keep chats. Without them the assistant still answers, but chats are not saved. Upstash also shares the rate limit across instances. LANGFUSE_SECRET_KEY and LANGFUSE_PUBLIC_KEY turn on Langfuse tracing.

Get started

1. Install the package with the peer dependencies it needs. It comes from the Evolve registry, which needs a one-time sign-in.

   pnpm add @evolve-framework/chat-backend graphql

   Requirements1

   graphql

   ^16.0.0 || ^17.0.0

For example:

import { ChatBackendModule } from "@evolve-framework/chat-backend/module";
import { createApplication } from "@evolve-framework/core";

const app = await createApplication({
  name: "chat",
  module: new ChatBackendModule({
    config: {
      id: "shop",
      storeContext: { storeKey: "nl", locale: "nl-NL", currency: "EUR" },
      model: {
        provider: "openai",
        modelId: "gpt-5.5",
        apiKey: process.env.OPENAI_API_KEY ?? "",
      },
      mcp: { endpoint: "http://localhost:6000/mcp", transport: "http" },
      systemPrompt: "You are the shopping assistant of our shop.",
      auth: {
        cookies: {
          token: "userToken",
          refreshTokenFlag: "userRefreshTokenExists",
        },
        jwtCookie: "userData",
        validateAuthenticated: false,
        firstNameClaim: "values.firstName",
      },
      productUrls: { productUrlPattern: "/p/{slug}" },
      allowedOrigins: ["https://www.example.com"],
    },
  }),
  http: { address: { host: "0.0.0.0", port: 8080 } },
});

await app.start();

Modules

ChatBackendModule

Chat-backend HTTP module.

Registers the chat API routes on the consuming service's Fastify app. The handlers are Web-standard (Request → Response); a thin adapter bridges each one to Fastify, including streaming chat responses.

• POST /api/chat streams an answer, with tools from config.mcp or a GraphQL query tool for config.graphql. It allows 20 requests per minute per IP address.
• GET /api/chat and DELETE /api/chat load and delete the shopper's chat.
• GET /api/chats lists the shopper's chats.
• GET /api/widget/config serves the public settings the widget reads.
• GET /healthcheck answers 200, unless registerHealthcheck is false.

The module reads a few settings from the environment. Chats are stored in Redis with REDIS_URL, or in Upstash with UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN; without either, the chat routes other than POST /api/chat answer 503. Upstash also shares the rate limit across instances, which is otherwise kept in memory. LANGFUSE_SECRET_KEY and LANGFUSE_PUBLIC_KEY turn on Langfuse tracing.

import { ChatBackendModule } from "@evolve-framework/chat-backend";

new ChatBackendModule(options: ChatBackendModuleOptions)

Option	Type	Description
config	ChatBackendConfig	The resolved single-tenant configuration for this deployment.
componentName?	string	Reported by GET /healthcheck. Defaults to "chat-backend".
basePath?	string	Mount all API routes under this path prefix (e.g. "/brand-a"). Defaults to "" (root). Required when combining multiple ChatBackendModule instances in one process (e.g. via CompositeModule) so their routes don't collide. A missing leading slash is added and a trailing slash is trimmed, so "brand-a", "/brand-a", and "/brand-a/" all mount at /brand-a.
registerHealthcheck?	boolean	Whether to register GET /healthcheck for this instance. Defaults to true. When combining multiple ChatBackendModule instances in one process, set this to false on all but one so there's a single canonical healthcheck for the container.

ChatBackendConfig16The fields of config.

Field	Type	Description
id	string	Stable identifier for this deployment (used in Redis keys + telemetry).
storeContext	StoreContext	Default store context.
auth	AuthConfig	Authentication configuration.
model	ModelConfig	The model this deployment uses: its provider, the modelId for that provider's SDK, the apiKey, and (for Azure) an endpoint and optional apiVersion. See ModelConfig.
systemPrompt	string	System prompt for the AI.
productUrls	ProductUrls	Product URL patterns.
allowedOrigins	string[]	Origins allowed to call this backend (widget host sites).
locales?	LocaleOption[]	Available locales (optional, for locale switching).
graphql?	GraphQLConfig	GraphQL configuration (optional when using MCP).
schemaPath?	string	Path to a GraphQL schema file, injected into {{SCHEMA}} of the prompt.
mcp?	MCPConfig	MCP server configuration (replaces the GraphQL query tool when set).
localTools?	Record<string, unknown>	Extra tools merged alongside the MCP/GraphQL tools, e.g. client-rendered widget tools whose execute transforms tool input into a render payload. Merged last, so a local tool wins a name collision with a discovered tool.
maxToolSteps?	number	Overrides the default 5-step tool-call cap for multi-step tool flows.
stopOnToolNames?	string[]	Tool names after whose call the run stops so the UI can collect user input; the tool still executes.
transformTools?	ToolTransform	Runs once per request after MCP/GraphQL discovery and the localTools merge; return the tool set the model sees (wrap, cap or drop tools).
branding?	Branding	Branding fallbacks (optional — the storefront owns presentation).

Extends AbstractModule<unknown>.

Entry points

@evolve-framework/chat-backend	ChatBackendModule and all configuration types18 exports
@evolve-framework/chat-backend/module	ChatBackendModule, the HTTP module with the chat API, and its option types2 exports
@evolve-framework/chat-backend/config	ChatBackendConfig and the types it is built from, for a service that builds its configuration in a separate file14 exports

Not in this package

The chat UI: that is the embeddable chat widget. Loading the configuration and its secrets from the environment is up to the service. The MCP server or GraphQL API that the tools call runs elsewhere.