Language Models Guide

Pick, inspect, and configure language models for your agents. Use this guide when you are selecting models, balancing cost versus latency, or migrating from provider-specific IDs to managed catalog entries.

Model coverage per interface lives in the AIP capability matrix. Highlights: selecting by language_model_id and applying runtime overrides are fully supported via REST/SDK, while dedicated CLI flags are still pending—use export/import or SDK helpers for those flows.

Discover Models

When to use: Inventory available providers or confirm new deployments before routing traffic.

Python SDK

from glaip_sdk import Client

client = Client()
for model in client.list_language_models():
    print(model["id"], model["name"], model.get("provider"))

CLI

REST

The response includes provider, model name, optional base URL, and any capability flags (context length, streaming support, etc.).

Assign Models to Agents

When to use: Update default models or move from legacy provider/model pairs to catalog-managed IDs.

The SDK supports three ways to specify models, from simplest to most flexible:

1. Model Constants (Recommended)

Use typed constants for IDE autocomplete, validation, and to avoid typos. These resolve to the standardized provider/model format.

from glaip_sdk import Agent
from glaip_sdk.models import OpenAI, DeepInfra, Anthropic, Google

# Using OpenAI models
agent = Agent(
    name="analysis",
    instruction="You are a precise analyst.",
    model=OpenAI.GPT_5_NANO,  # Resolves to "openai/gpt-5-nano"
)

# Using DeepInfra models
agent = Agent(
    name="research",
    instruction="You are a research assistant.",
    model=DeepInfra.QWEN3_30B_A3B,  # Resolves to "deepinfra/Qwen/Qwen3-30B-A3B"
)

# Using Anthropic models
agent = Agent(
    name="creative",
    instruction="You are a creative writer.",
    model=Anthropic.CLAUDE_SONNET_4_0,  # Resolves to "anthropic/claude-sonnet-4-0"
)

Available constants:

Provider	Import	Examples
OpenAI	from glaip_sdk.models import OpenAI	GPT_5_NANO, GPT_5_MINI, GPT_5, GPT_4O
Anthropic	from glaip_sdk.models import Anthropic	CLAUDE_SONNET_4_0, CLAUDE_OPUS_4_0
Google	from glaip_sdk.models import Google	GEMINI_2_5_FLASH, GEMINI_2_5_PRO
Azure OpenAI	from glaip_sdk.models import AzureOpenAI	GPT_4O, GPT_4O_MINI
DeepInfra	from glaip_sdk.models import DeepInfra	QWEN3_30B_A3B, KIMI_K2_INSTRUCT
DeepSeek	from glaip_sdk.models import DeepSeek	DEEPSEEK_CHAT
AWS Bedrock	from glaip_sdk.models import Bedrock	CLAUDE_SONNET_4_20250514_V1_0

The default model is OpenAI.GPT_5_NANO.

2. String Format (Direct)

Use the standardized provider/model format directly as a string. Use this when you need a model not in the constants, or when working in environments without IDE support.

# OpenAI: openai/<model-name>
agent = Agent(
    name="analysis",
    instruction="You are a precise analyst.",
    model="openai/gpt-5",
)

# DeepInfra: deepinfra/<org>/<model-name>
agent = Agent(
    name="research",
    instruction="You are a research assistant.",
    model="deepinfra/Qwen/Qwen3-30B-A3B",
)

# Anthropic: anthropic/<model-name>
agent = Agent(
    name="creative",
    instruction="You are a creative writer.",
    model="anthropic/claude-sonnet-4-0",
)

# CLI also uses the string format
aip agents create \
  --name analysis \
  --instruction "You are a precise analyst." \
  --model openai/gpt-5

Format patterns:

• OpenAI: openai/<model> → openai/gpt-5, openai/gpt-4o

• DeepInfra: deepinfra/<org>/<model> → deepinfra/Qwen/Qwen3-30B-A3B

• DeepSeek: deepseek/<org>/<model> → deepseek/deepseek-ai/DeepSeek-V3.1

• Anthropic: anthropic/<model> → anthropic/claude-sonnet-4-0

• Google: google/<model> → google/gemini-2.5-flash

• Azure OpenAI: azure-openai/<model> → azure-openai/gpt-4o

Invalid formats (missing /) raise a ValueError with suggestions to use model constants.

Common routing issues

Symptom	Likely cause	Fix
Agent still uses the old model	Cached agent config or CLI not updated.	Re-run aip agents update and confirm the agent payload includes the new language_model_id.
400 Unknown model errors	Model ID not available in the target environment.	List models in that environment or request the provider be deployed there.
High latency after switching models	New model has different performance profile.	Adjust timeout_seconds in agent_config for latency-sensitive runs.
Streaming stops mid-response	Model lacks streaming support.	Disable streaming for that agent or choose a streaming-capable model.

Advanced: Custom Model Configuration

For custom endpoints, credentials, and hyperparameters not covered by the standard format, use the Model class:

from glaip_sdk import Agent
from glaip_sdk.models import Model

# Create agent using custom Moonshot Kimi K2.5 model
agent = Agent(
    name="kimi-agent",
    instruction="You are a helpful AI assistant powered by Moonshot Kimi K2.5.",
    model=Model(
        id="custom/kimi-k2.5",
        base_url="https://api.moonshot.ai/v1",
        credentials="sk-xxxx",  # Your API key
        hyperparameters={
            "temperature": 1.0,
            "max_tokens": 32768,
            "top_p": 0.95,
        },
    ),
)

Model Configuration Breakdown:

• id: Model identifier in provider/model format. Use "custom/<name>" for custom providers not in built-in constants.

• base_url: The API endpoint URL for the model provider (e.g., "https://api.moonshot.ai/v1").

• credentials: API key as a string, or a dict with additional credential fields.

• hyperparameters: Model-specific parameters like temperature, max_tokens, top_p, etc.

Custom DeepInfra Model with Credentials

from glaip_sdk import Agent
from glaip_sdk.models import Model

agent = Agent(
    name="custom-deepinfra",
    instruction="You are a helpful assistant.",
    model=Model(
        id="deepinfra/Qwen/Qwen3-30B-A3B",
        credentials="your-deepinfra-api-key",  # Override default credential lookup
        hyperparameters={
            "temperature": 0.7,
            "max_tokens": 4096,
        },
    ),
)

Credential Precedence: When using Model with credentials, the SDK uses this priority order:

1. Explicit credentials in Model.credentials (highest priority)

2. Environment variables (e.g., DEEPINFRA_API_KEY, OPENAI_API_KEY)

3. Credential files or default locations

For providers like DeepInfra and DeepSeek, the SDK automatically resolves the base_url from built-in configurations, so you don't need to specify it unless using a custom endpoint.

Local vs Remote Execution

When to use: Understand how model configuration behaves when switching between local execution (via agent.run()) and remote execution (via agent.deploy() and server-side runs).

How Model Configuration Works

Execution Mode	Model Configuration	Behavior
Local (agent.run())	Model object with base_url, credentials, hyperparameters	Full configuration is used directly by aip_agents
Local	String like "deepinfra/Qwen/Qwen3-30B-A3B"	SDK resolves base_url from built-in provider mappings; credentials from env vars
Remote (agent.deploy())	Model object or string	Agent is deployed to AIP server; server uses its configured language_model_id mappings
Remote	language_model_id="uuid"	Server looks up model in its catalog; ignores local config

Example: Local Execution with AIP Server Models

When a model exists on the AIP server (has a language_model_id), you can still use it locally:

from glaip_sdk import Agent

# This works locally IF the model is available locally
# (API key and base_url resolved automatically for known providers)
agent = Agent(
    name="research",
    instruction="You are a research assistant.",
    model="deepinfra/Qwen/Qwen3-30B-A3B",
)

# Local execution - uses local credentials and base_url
result = agent.run("Research quantum computing")

Built-in Provider Configurations: The SDK includes default base_url mappings for:

• deepinfra → https://api.deepinfra.com/v1/openai

• deepseek → https://api.deepseek.com

For these providers, you only need to set the corresponding environment variable (DEEPINFRA_API_KEY, DEEPSEEK_API_KEY) or pass credentials explicitly.

Example: Custom Model Not in AIP Server

For models not configured on the AIP server, use the Model class with full configuration:

from glaip_sdk import Agent
from glaip_sdk.models import Model

# Custom endpoint not in AIP server catalog
agent = Agent(
    name="custom-model",
    instruction="You are helpful.",
    model=Model(
        id="custom/my-model",
        base_url="https://my-custom-api.example.com/v1",
        credentials="my-api-key",
    ),
)

# Works locally - credentials and base_url are used directly
result = agent.run("Hello!")

# Deploy to AIP server - requires server-side configuration first
# (See "Adding Language Models to AIP Server" below)
# agent.deploy()  # May fail if model not registered on server

Switching Between Local and Remote

When you call agent.deploy(), the agent is synchronized with the AIP server. The server manages its own model catalog via language_model_id mappings:

from glaip_sdk import Agent

agent = Agent(
    name="my-agent",
    instruction="You are helpful.",
    model="openai/gpt-5",
)

# Local execution - uses local OPENAI_API_KEY
result = agent.run("Hello!")

# Remote execution - deploy first, then server uses its configured mappings
agent.deploy()  # Server resolves "openai/gpt-5" to its language_model_id
result = agent.run("Hello!")  # Now runs on AIP server

Configuration Isolation:

• Local execution uses your local environment: base_url, credentials from Model or env vars

• Remote execution uses AIP server's configuration: language_model_id mappings, server-side credentials

• Changes to local model config don't affect deployed agents until you call agent.deploy() again

Using Custom Models with AIP Server

When to use: You want to use a custom model in remote execution (via agent.deploy()) that's not in the AIP server's catalog.

Option 1: Local Execution (Recommended for Custom Models)

For models not in the AIP server catalog, the simplest approach is to use local execution with the Model class:

from glaip_sdk import Agent
from glaip_sdk.models import Model

agent = Agent(
    name="kimi-agent",
    instruction="You are helpful.",
    model=Model(
        id="custom/kimi-k2.5",
        base_url="https://api.moonshot.ai/v1",
        credentials="sk-xxxx",
        hyperparameters={
            "temperature": 1.0,
            "max_tokens": 32768,
        },
    ),
)

# Run locally - no server registration needed
result = agent.run("Hello!")

This works immediately without any server-side configuration. The SDK passes your credentials and configuration directly to the local runner.

Option 2: Register Model on AIP Server

To use a custom model in remote execution (via agent.deploy()), the model must be registered on the AIP server first.

Check available models:

# List all models configured on the AIP server
aip models list

Register a new model:

The CLI does not currently support creating language models directly. You have two options:

Option A: Use the REST API (requires admin/master API key)

curl --location "$AIP_API_URL/language-models/" \
  --header "Content-Type: application/json" \
  --header "X-API-Key: $AIP_MASTER_API_KEY" \
  --data '{
    "provider": "custom",
    "name": "kimi-k2.5",
    "hyperparameters": {
      "temperature": 0.5
    },
    "base_url": "https://api.moonshot.ai/v1",
    "credentials": "your-api-key-here"
  }'

Required fields:

• provider: Provider identifier (e.g., "custom", "openai-compatible")

• name: Model name identifier

Optional fields:

• base_url: API endpoint URL for the provider

• credentials: API key or authentication token

• hyperparameters: Default hyperparameters like temperature, max_tokens, etc.

• capabilities: Feature flags (streaming, function calling, etc.) - configured server-side

• context_length: Maximum context window size - configured server-side

Option B: Ask your admin to register the model with details like:

• provider: Provider identifier (e.g., "custom", "openai-compatible")

• name: Model name (e.g., "kimi-k2.5")

• base_url: API endpoint URL (e.g., "https://api.moonshot.ai/v1")

• credentials: API key for authentication

• hyperparameters: Default model settings (optional)

Once registered, you can use the model by name in remote execution:

from glaip_sdk import Agent

agent = Agent(
    name="kimi-agent",
    instruction="You are helpful.",
    model="custom/kimi-k2.5",  # Use provider/name format
)

# Deploy and run on AIP server
agent.deploy()
result = agent.run("Hello!")

When to register on server:

• You need remote execution (running agents on AIP server infrastructure)

• You want to share the model across your team without sharing API keys

• You want centralized credential management (API keys stored on server, not locally)

When to use local execution:

• Quick prototyping and testing

• Models with local-only access or custom endpoints

• When you don't have admin access to register models

Related Documentation

• Agents guide — apply models during creation and updates, plus runtime overrides.

• Automation & scripting — script model discovery and promotion across environments.

• CLI commands reference — view available models with aip models list.