Skip to main content
A hosted provider supplies inference. The ngram runtime still owns the Entity: its identity, memory stores, relationships, tools, scheduling, and spatial connections.

​
Start with the hosted profile

Choose the provider and exact model IDs your account can use. Store the API key privately when prompted. You do not need Ollama, a GPU, Railway, or a tunnel for this setup. For an OpenAI route, the non-secret environment configuration can look like:
Supply OPENAI_API_KEY in your private environment. The model ID above is an ngram configuration example; access and provider capabilities depend on your account.

​
Chat and memory need separate models

Chat generates responses. Embeddings turn text into vectors for memory retrieval. A hosted profile needs both capabilities, or an explicitly retained existing embedding route. The lab and runtime brain selector probe embeddings and check the configured vector width before accepting a fully hosted profile. If the runtime switch fails readiness checks, it keeps the previous provider.
Equal vector dimensions do not make different embedding models semantically compatible. For an existing Entity, keep the embedding model stable or plan a memory migration. A successful width check alone does not re-embed old memories.

​
Provider presets

The runtime recognizes openai, anthropic, gemini, openrouter, xai, groq, together, fireworks, mistral, deepseek, venice, and custom, in addition to local and remote_gateway. Presets define transport settings; they do not certify every model or endpoint capability. Native OpenAI uses the Responses transport. The other hosted presets use the runtime’s OpenAI-compatible transport, so verify chat, tool calling, and embeddings with the endpoint you choose. Default credential variable names are listed in the environment reference.

​
Base URLs

Leave NGRAM_INFERENCE_BASE_URL unset to use a managed provider’s default endpoint.
  • For custom, the URL is the complete API prefix, such as https://inference.example.com/v1.
  • Managed providers recognize their normal API suffix or a complete /v1 override and avoid appending it twice.
  • Gemini, Groq, and Fireworks have provider-specific default API prefixes. Prefer the preset unless you intend to override it.
Set NGRAM_INFERENCE_PASS_NUM_CTX=false for hosted APIs. num_ctx is an Ollama-specific option, not a portable hosted context setting.

​
Change the model without changing the Entity

The spatial brain selector can switch between local, private-gateway, and hosted routes. A switch updates the shared Entity’s inference consumers; it does not create a new identity. NGRAM_INFERENCE_MODEL overrides the chat model IDs from Entity YAML at startup. Restart-time environment configuration and an in-memory runtime switch are different persistence mechanisms; configure your deployed variables for the provider you want after restart. Use Pause inference to stop paid model use. Selecting a quiet chat view or closing the shell does not stop an always-on worker.