Language Model (LM) Invoker

gllm-inference | Tutorial: Language Model (LM) Invoker| Use Case: Utilize Language Model Request Processor | API Reference

What’s an LM Invoker?

The LM invoker is a unified interface designed to help you interact with language models to generate outputs based on the provided inputs. In this tutorial, you'll learn how to invoke a language model using OpenAILMInvoker in just a few lines of code. You can also explore other types of LM Invokers, available here.

Prerequisites

This example specifically requires completion of all setup steps listed on the Prerequisites page.

Installation

Linux, macOS, or Windows WSL

# you can use a Conda environment
pip install --extra-index-url https://oauth2accesstoken:$(gcloud auth print-access-token)@glsdk.gdplabs.id/gen-ai-internal/simple/ gllm-inference

Windows Powershell

Windows Command Prompt

Quickstart

Initialization and Invoking

Let’s jump into a basic example using OpenAILMInvoker. We’ll ask the model a simple question and print the response.

import asyncio
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM

lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO)
response = asyncio.run(lm_invoker.invoke("What is the capital city of Indonesia?"))
print(f"Response: {response}")

Output:

Response: Jakarta.

Understanding LM Invoker Output Type

Starting v0.6.0, the output of LM Invoker will be LMOutput object only.

Depending on how you configure the LM Invoker, the result of invoke(...) may be either a plain string or an LMOutput object:

1. String → returned when you don’t request any extra features. Example:

   response = asyncio.run(lm_invoker.invoke("What is the capital of Indonesia?"))
   print(response)  # "Jakarta."

2. LMOutput → returned whenever a response contains more than just plain text, such as when features like Structured Output, Tool Calling, Reasoning, Citations (Web Search), Output Analytics, or Code Execution are used. In these cases, you can still access the generated string through response.response, while also taking advantage of the additional attributes exposed by the enabled feature

Below is the key attributes of LMOutput based on which features of LM Output is used:

Feature	Attribute	Type
All	response	str
Tool Calling	tool_calls	list[ToolCall]
Structured Output	structured_output	dict | BaseModel | None
Output Analytics	token_usage	TokenUsage | None
Output Analytics	duration	float | None
Output Analytics	finish_details	dict
Reasoning	reasoning	list[Reasoning]
Language Model (LM) Invoker	citations	list[Chunk]
Language Model (LM) Invoker	code_exec_results	list[CodeExecResult]

---

Message Roles

Modern LMs understand context better when you structure inputs like a real conversation. That’s where message roles come in. You can simulate multi-turn chats, set instructions, or give memory to the model through a structured message format.

Example 1: Passing a system message

import asyncio
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_inference.schema import Message

messages = [
    Message.system("Talk like a pirate."),
    Message.user("Hi, there! How are you doing?")
]

lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO)
response = asyncio.run(lm_invoker.invoke(messages))
print(f"Response: {response}")

Output:

Response: Ahoy, matey! I be doin' well, savvy? How be ye farin' on this fine day?

Example 2: Simulating a multi-turn conversation

import asyncio
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_inference.schema import Message

messages = [
    Message.system("You are a helpful assistant."),
    Message.user("What is the capital of France?"),
    Message.assistant("The capital of France is Paris."),
    Message.user("What about Indonesia?"),
]

lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO)
response = asyncio.run(lm_invoker.invoke(messages))
print(f"Response: {response}")

Output:

Response: The capital of Indonesia is Jakarta.

---

Multimodal Input

Our LM Invokers supports attachments (images, documents, etc.). This lets you send rich content and ask the model to analyze or describe them.

Loading Attachments

An Attachment is a content object that can be loaded in the following ways:

• using a remote file (URL)

• using a local file

• using a data URL

• using raw bytes

We can load the attachment in the following ways:

from gllm_inference.schema import Attachment

# From remote URL
image = Attachment.from_url("https://example.com/image.png")

# From local file
image = Attachment.from_path("path/to/image.jpeg")

# From Data URL
image = Attachment.from_data_url("data:image/jpeg;base64,<base64_encoded_image>")

# From bytes
image = Attachment.from_bytes(b"<image_bytes>")

Example 1: Describe an image

import asyncio
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_inference.schema import Attachment

image = Attachment.from_path("path/to/dog.jpeg")

lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO)
response = asyncio.run(lm_invoker.invoke(["What is this?", image]))
print(f"Response: {response}")

Output:

Response: A cute golden retriever puppy.

---

Example 2: Analyze a PDF

import asyncio
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_inference.schema import Attachment

URL = "https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf"
document = Attachment.from_url(URL)

lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO)
response = asyncio.run(lm_invoker.invoke(["What is the title of this file?", document]))
print(f"Response: {response}")

Output:

Response: This is the scientific paper titled "Attention Is All You Need".

---

Supported Attachment Types

Each LM might support different types of inputs. As of now, OpenAILMInvoker supports image and documents. You can find more about supported type for each LM Invoker here.

---

Structured Output

In many real-world applications, we don't just want natural language responses — we want structured data that our programs can parse and use directly.

You can define your expected output using:

1. A Pydantic BaseModel class (recommended).

2. A JSON schema dictionary compatible with Pydantic's schema format.

When structured output is enabled, the output will be stored in the structured_output attribute of the response. The output type depends on the input schema:

1. Pydantic instance → The output will be a Pydantic BaseModel instance.

2. JSON schema dict → The output will be a Python dictionary.

Using a Pydantic BaseModel (Recommended)

You can define your expected response format as a Pydantic class. This ensures strong type safety and makes the output easier to work with in Python.

import asyncio
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from pydantic import BaseModel

class Animal(BaseModel):
    name: str
    size: str
    diet: str
    color: list[str]
    legs: int

lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO, response_schema=Animal)
response = asyncio.run(lm_invoker.invoke("Describe a chicken!"))
print(f"Response: {response}")

Output:

Response: LMOutput(
    structured_output=Animal(
        name='Chicken',
        size='Medium', 
        diet='Omnivore', 
        color=['varies by breed'], 
        legs=2,
    )
)

Using a JSON Schema Dictionary

Alternatively, you can define the structure using a JSON schema dictionary.

import asyncio
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM

animal_schema = {
    "properties": {
        "color": {
            "items": {"type": "string"},
            "title": "Color",
            "type": "array"
        },
        "diet": {"title": "Diet", "type": "string"},
        "legs": {"title": "Legs", "type": "integer"},
        "name": {"title": "Name", "type": "string"},
        "size": {"title": "Size", "type": "string"}
    },
    "required": ["name", "size", "diet", "color", "legs"],
    "title": "Animal",
    "type": "object",
}

lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO, response_schema=animal_schema)
response = asyncio.run(lm_invoker.invoke("Describe a chicken!"))
print(f"Response: {response}")

Output

Response: LMOutput(
    structured_output={
        'color': ['reddish-brown feathers', 'white underparts'], 
        'diet': 'Omnivore; seeds, grains, insects, greens', 
        'legs': 2, 
        'name': 'Chicken', 
        'size': 'Medium',
    }
)

If JSON schema is used, it must still be compatible with Pydantic's JSON schema, especially for complex schemas. For this reason, it is recommended to create the JSON schema using Pydantic's model_json_schema method.

animal_schema = Animal.model_json_schema()

---

Tool Calling

Tool calling means letting a language model call external functions to help it solve a task. It allows the AI to interact with external functions and APIs during the conversation, enabling dynamic computation, data retrieval, and complex workflows.

Think of it as:

The LM is smart at reading and reasoning, but when it needs to calculate or get external data, it picks up the phone and calls your "tool".

Tool Definition

We can easily convert a function into tools using the tool decorator.

from gllm_core.schema.tool import tool

@tool
def add(a: int, b: int) -> int:
    """Add two numbers.
    
    Args:
        a (int): The first number.
        b (int): The second number.
        
    Returns:
        int: The added number.
    """
    return a + b

LM Invocation with Tool

Let's try to integrate the above tool to our LM invoker!

import asyncio
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_core.schema.tool import tool

@tool
def add(a: int, b: int) -> int:
    """Add two numbers.
    
    Args:
        a (int): The first number.
        b (int): The second number.
        
    Returns:
        int: The added number.
    """
    return a + b

lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO, tools=[add])
response = asyncio.run(lm_invoker.invoke("What is 25 + 15?"))
print(f"Response: {response}")

Output:

Response: LMOutput(
    tool_calls=[ToolCall(id='call_123', name='add', args={'a': 25, 'b': 15})]
)

When the LM Invoker is invoked with tool calling capability, the model will return the tool calls. In this case, we still need to execute the tools and feed the result back to the LM invoker ourselves. If you'd like to handle this looping process automatically, please refere to the LM Request Processor component.

---

Native Tools

Native tools are a specific set tools that allow the language model to execute certain built-in capabilities during the invocation, enabling dynamic computation, data retrieval, and complex workflows. Similar to the user-defined tools, the native tools can be enabled by passing them through the LM invoker's tools parameter.

Each type of native tools is only available for certain LM invokers. Please find the available native tools below:

1. Code interpreter — Writes and runs Python code in a sandboxed environment.

2. Web search — Searches the web for relevant information.

3. MCP Server — Uses remote MCP servers to give models new capabilities.

---

Output Analytics

Output analytics enables you to collect detailed metrics and insights about your language model invocations. When output analytics is enabled, the response includes the following extra attributes:

1. token_usage : Input and output token counts.

2. duration : Time taken to generate the response.

3. finish_details: Additional details about how the generation finished.

To enable output analytics, simply need to set the output_analytics parameter to True.

import asyncio
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM

lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO, output_analytics=True)
response = asyncio.run(lm_invoker.invoke("What is the capital of France?"))
print(f"Response: {response}")

Output:

Response: LMOutput(
    response=Paris.,
    token_usage=input_tokens=13 output_tokens=8,
    duration=2.211723566055298,
    finish_details={'status': 'completed', 'incomplete_details': {'reason': None}}
)

---

Retry & Timeout

Retry & timeout functionality provides robust error handling and reliability for language model interactions. It allows you to automatically retry failed requests and set time limits for operations, ensuring your applications remain responsive and resilient to network issues or API failures.

Retry & timeout can be configured via the RetryConfig class' parameters:

1. max_retries: Maximum number of retry attempts (defaults to 3 maximum retry attempts).

2. timeout: Maximum time in seconds to wait for each request (defaults to 30.0 seconds). To disable timeout, this parameter can be set to 0.0 second.

You can also configure other parameters available here. Now let's try to apply it to our LM invoker!

import asyncio
from gllm_core.utils.retry import RetryConfig
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM

retry_config = RetryConfig(max_retries=3, timeout=100)
lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO, retry_config=retry_config)
response = asyncio.run(lm_invoker.invoke("What is the capital of France?"))
print(f"Response: {response}")

---

Extra Capabilities

Some LM invokers also provide additional capabities that are useful in certain cases:

1. Batch Invocation — To manage batch requests for cheaper but slower invocations.

2. File Management — To manage uploaded files in their server side. These files can then be used as inputs during invocations.

3. Data Store Management — To manage built-in data stores to be used as internal knowledge base. This allows the LM invoker to perform built-in RAG (Retrieval-Augmented Generation).

---

Reasoning

Certain language model providers and models supports reasoning. When reasoning is used, models think before they answer, producing a long internal chain of thought before responding to the user. Reasoning allows model to perform advanced tasks such as complex problem solving, coding, scientific reasoning, and multi-step planning for agentic workflows.

However, it's important to note that the way reasoning works varies between providers. Therefore, it is important to check how each class handled reasoning before using them.

In the case of OpenAILMInvoker, reasoning can be done by using reasoning models (Often referred as the o-series models). Then, we can set the reasoning_summary parameter to output the reasoning tokens.

The reasoning tokens will be stored in the reasoning attribute of the output.

import asyncio
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM

lm_invoker = OpenAILMInvoker(OpenAILM.O4_MINI, reasoning_summary="detailed")
response = asyncio.run(lm_invoker.invoke("Solve x^2 + 2x + 1 = 0"))

for item in response.reasoning:
    print(f"=== Reasoning ===\n{item.reasoning}\n")

print(f"=== Response ===\n{response.response}\n")

Output:

=== Reasoning ===
**Solving the quadratic equation**

The user wants to solve the equation x² + 2x + 1 = 0. 
I see that this can be factored as (x + 1)² = 0, which gives a double root at x = -1. 
So, the solution is x = -1, with multiplicity two. 
I also checked the discriminant, which confirms this: b² - 4ac = 4 - 4 = 0. 
It might be useful to show the steps, including factorization and mention that it's a perfect square with a repeated root.

=== Response ===
The equation factors as  
x² + 2x + 1 = (x + 1)² = 0  

Hence the only solution (a double root) is  
x = –1.