Produce Consistent Output from LM

This guide will walk you through creating structured output responses using LM Request Processor (LMRP) with response schemas.

Structured output allows you to receive LM responses in a predefined, consistent format (Pydantic BaseModel/JSON). Instead of getting unstructured text, you get validated Python objects that are ready to use in your application.

Prerequisites

This example specifically requires:

1. Completion of all setup steps listed on the Prerequisites page.

2. A working OpenAI API key configured in your environment variables.

You should be familiar with these concepts and components:

1. Language Model (LM) Invoker

2. LM Request Processor (LMRP)

3. Prompt Builder

4. Basic understanding of Pydantic models and async Python programming

View full project code on GitHub

Installation

Linux, macOS, or Windows WSL

# you can use a Conda environment
pip install --extra-index-url "https://oauth2accesstoken:$(gcloud auth print-access-token)@glsdk.gdplabs.id/gen-ai-internal/simple/" gllm-inference

Windows Powershell

Windows Command Prompt

You can either:

1. You can refer to the guide whenever you need explanation or want to clarify how each part works.

2. Follow along with each step to recreate the files yourself while learning about the components and how to integrate them.

Both options will work—choose based on whether you prefer speed or learning by doing!

Project Setup

1

Environment Configuration

Ensure you have a file named .env in your project directory with the following content:

OPENAI_API_KEY="<YOUR_OPENAI_API_KEY>"

Replace <YOUR_OPENAI_API_KEY> with your actual OpenAI API key.

---

Option 1: Using LM Invoker's Response Schema

1) Define Your Response Schema

The response schema defines the exact structure you want the AI to return. We'll use Pydantic models to define this structure:

1

Import Required Libraries

Start by importing the necessary dependencies:

from pydantic import BaseModel
from typing import List
from dotenv import load_dotenv
import asyncio

load_dotenv()

2

Create Your Pydantic Models

Define the structure for individual activities and the complete response:

class Activity(BaseModel):
    type: str
    activity_location: str
    description: str

class ActivityList(BaseModel):
    location: str
    activities: List[Activity]

🧠 These models define exactly what fields the AI response must include and their data types.

2) Configure the LM Invoker

The LM invoker handles communication with the language model and enforces the response schema:

1

Set up the LM Invoker with Response Schema

from gllm_inference.lm_invoker import OpenAILMInvoker

lm_invoker = OpenAILMInvoker(
    model_name="gpt-4o-mini",
    response_schema=ActivityList  # This enforces structured output
)

The response_schema parameter ensures the AI response matches your Pydantic model exactly.

The response schema acts as a contract between your application and the AI model, guaranteeing consistent output structure.

3) Create the Prompt Builder

The prompt builder formats your prompts consistently:

1

Define Your Prompt Templates

from gllm_inference.prompt_builder import PromptBuilder

system_template = "You are a helpful assistant who specializes in recommending activities."
user_template = "{question}"

prompt_builder = PromptBuilder(
    system_template=system_template,
    user_template=user_template
)

🧠 The {question} placeholder will be replaced with actual user input when processing requests.

4) Build the LM Request Processor

The LM request processor combines your prompt builder and LM invoker into a complete processing pipeline:

1

Create the Request Processor

from gllm_inference.request_processor import LMRequestProcessor

lm_request_processor = LMRequestProcessor(
    prompt_builder=prompt_builder,
    lm_invoker=lm_invoker,
)

This creates a complete pipeline that will:

1. Format your prompt using the prompt builder

2. Send it to the LM invoker with schema enforcement

3. Return structured, validated results

🧠 The LM Request Processor automatically handles the entire workflow, making structured output generation seamless.

5) Process Requests and Get Structured Output

Now you can process requests and receive structured responses:

1

Process a Request

response = asyncio.run(lm_request_processor.process(
    prompt_kwargs={
        "question": "I want to go to Tokyo, Japan. What should I do?"
    }
))

# Access the structured output
print(response.structured_output)

2

Expected Output Structure

The response will be a validated ActivityList object:

ActivityList(
    location="Tokyo, Japan",
    activities=[
        Activity(
            type="Cultural",
            activity_location="Senso-ji Temple",
            description="Visit Tokyo's oldest temple in Asakusa district"
        ),
        Activity(
            type="Shopping",
            activity_location="Shibuya Crossing",
            description="Experience the world's busiest pedestrian crossing"
        ),
        # ... more activities
    ]
)

Notice how every field matches exactly what was defined in your Pydantic models - no parsing or validation needed!

3

Access Individual Fields

You can access specific data from the structured response:

# Get the location
location = response.structured_output.location
print(f"Destination: {location}")

# Iterate through activities
for activity in response.structured_output.activities:
    print(f"- {activity.type}: {activity.description}")
    print(f"  Location: {activity.activity_location}")

Option 2: Using JSON Output Parser

This approach uses the JSON Output Parser to handle structured output parsing after the LM generates a response. Instead of enforcing the schema at the LM level, it relies on prompt instructions and post-processing.

1) Define Your Response Schema

The response schema definition remains the same as Option 1:

1

Import Required Libraries

from pydantic import BaseModel
from typing import List
from dotenv import load_dotenv
from gllm_inference.output_parser import JSONOutputParser
import asyncio

load_dotenv()

2

Create Your Pydantic Models

class Activity(BaseModel):
    type: str
    activity_location: str
    description: str

class ActivityList(BaseModel):
    location: str
    activities: List[Activity]

🧠 The same Pydantic models work for both approaches - the difference is in how they're applied.

2) Configure the JSON Output Parser

Create the output parser that will handle the JSON parsing and validation:

1

Set up the JSON Output Parser

output_parser = JSONOutputParser()

The JSON Output Parser automatically handles JSON parsing and can validate against Pydantic models.

3) Configure the LM Invoker

Unlike Option 1, the LM invoker doesn't need a response schema parameter:

1

Set up the LM Invoker

from gllm_inference.lm_invoker import OpenAILMInvoker

lm_invoker = OpenAILMInvoker(model_name="gpt-4o-mini")

🧠 Notice there's no response_schema parameter - the structure is enforced through prompting and parsing.

4) Create the Prompt Builder with Schema Instructions

The prompt must instruct the model to return JSON in the expected format:

1

Define Your Prompt Templates with Schema

from gllm_inference.prompt_builder import PromptBuilder

system_template = """You are a helpful assistant who specializes in recommending activities. 
Return the response in JSON format with the schema: {schema}."""

user_template = "{question}"

prompt_builder = PromptBuilder(
    system_template=system_template,
    user_template=user_template
)

🧠 The {schema} placeholder will be filled with the actual JSON schema at runtime.

5) Build the LM Request Processor with Output Parser

Include the output parser in the request processor configuration:

1

Create the Request Processor

from gllm_inference.request_processor import LMRequestProcessor

lm_request_processor = LMRequestProcessor(
    prompt_builder=prompt_builder,
    lm_invoker=lm_invoker,
    output_parser=output_parser
)

This creates a pipeline that will:

1. Format your prompt with schema instructions

2. Send it to the LM invoker

3. Parse and validate the JSON response using the output parser

🧠 The output parser handles both JSON parsing and optional Pydantic model validation.

6) Process Requests with Schema Parameter

Pass the schema as a prompt parameter when processing requests:

1

Process a Request with Schema

response = asyncio.run(lm_request_processor.process(
    prompt_kwargs={
        "question": "I want to go to Tokyo, Japan. What should I do?",
        "schema": str(ActivityList.model_json_schema())
    }
))

# Access the parsed output
print(response)

2

Expected Output Structure

The response will contain parsed JSON data that matches your schema structure:

{
    "location": "Tokyo, Japan",
    "activities": [
        {
            "type": "Cultural",
            "activity_location": "Senso-ji Temple",
            "description": "Visit Tokyo's oldest temple in Asakusa district"
        },
        # ... more activities
    ]
}

The output is parsed JSON data, ready for further processing or conversion to Pydantic models if needed.

📂 Complete Guide Files

Option 1: Using LM Invoker's Response Schema

from dotenv import load_dotenv
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.request_processor import LMRequestProcessor
from pydantic import BaseModel
from typing import List
from gllm_inference.prompt_builder import PromptBuilder
import asyncio

load_dotenv()

# Define the response schema
class Activity(BaseModel):
    type: str
    activity_location: str
    description: str

class ActivityList(BaseModel):
    location: str
    activities: List[Activity]

# Define the LM invoker with response schema
lm_invoker = OpenAILMInvoker(model_name="gpt-4o-mini", response_schema=ActivityList)

# Define the prompt
system_template = "You are a helpful assistant who specializes in recommending activities."
user_template = "{question}"

prompt_builder = PromptBuilder(system_template=system_template, user_template=user_template)

# Define the LM request processor
lm_request_processor = LMRequestProcessor(
    prompt_builder=prompt_builder,
    lm_invoker=lm_invoker,
)

# Invoke the LM request processor
response = asyncio.run(lm_request_processor.process(
    prompt_kwargs={
        "question": "I want to go to Tokyo, Japan. What should I do?"
    }
))

print(response.structured_output)

Option 2: Using JSON Output Parser

from dotenv import load_dotenv
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.request_processor import LMRequestProcessor
from gllm_inference.output_parser import JSONOutputParser
from pydantic import BaseModel
from typing import List
from gllm_inference.prompt_builder import PromptBuilder
import asyncio

load_dotenv()

# Define the response schema
class Activity(BaseModel):
    type: str
    activity_location: str
    description: str

class ActivityList(BaseModel):
    location: str
    activities: List[Activity]

# Create output parser
output_parser = JSONOutputParser()

# Define the LM invoker (no response schema)
lm_invoker = OpenAILMInvoker(model_name="gpt-4o-mini")

# Define the prompt with schema instruction
system_template = "You are a helpful assistant who specializes in recommending activities. Return the response in JSON format with the schema: {schema}."
user_template = "{question}"

prompt_builder = PromptBuilder(system_template=system_template, user_template=user_template)

# Define the LM request processor with output parser
lm_request_processor = LMRequestProcessor(
    prompt_builder=prompt_builder,
    lm_invoker=lm_invoker,
    output_parser=output_parser
)

# Invoke the LM request processor with schema parameter
response = asyncio.run(lm_request_processor.process(
    prompt_kwargs={
        "question": "I want to go to Tokyo, Japan. What should I do?",
        "schema": str(ActivityList.model_json_schema())
    }
))

print(response)

---