LM Request Processor (LMRP)

gllm-inference | Tutorial: LM Request Processor (LMRP) | Use Case: Utilize Language Model Request Processor | API Reference

What’s an LMRP?

The LM Request Processor (LMRP) is an orchestrator module that wraps a prompt builder, an LM invoker, and optionally an output parser to perform end-to-end LM invocation in a single process. In this tutorial, you'll learn how to use the LMRequestProcessor in just a few lines of code.

Prerequisites

This example specifically requires completion of all setup steps listed on the Prerequisites page.

Installation

Linux, macOS, or Windows WSL

# you can use a Conda environment
pip install --extra-index-url https://oauth2accesstoken:$(gcloud auth print-access-token)@glsdk.gdplabs.id/gen-ai-internal/simple/ gllm-inference

Windows Powershell

Windows Command Prompt

Quickstart

Let’s jump into a basic example using LMRequestProcessor. This basic LMRP usage will only utilize a simple PromptBuilder and an OpenAILMInvoker.

import asyncio
from gllm_inference.prompt_builder import PromptBuilder
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_inference.request_processor import LMRequestProcessor

prompt_builder = PromptBuilder(
    system_template="Talk like a pirate.", 
    user_template="What is the capital city of Indonesia?",
)
lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO)
lm_request_processor = LMRequestProcessor(prompt_builder, lm_invoker)
response = asyncio.run(lm_request_processor.process())
print(f"Response: {response}")

Expected Output

Response: Arrr, the capital o' Indonesia be Jakarta.

Using Output Parser

Optionally, the LMRP can also utilize an output parser. In this case, the LMRP must be configured to output a response with a format that is compatible with the output parser (e.g. through the prompt template). In the example below, we're instructing the language model to answer in JSON format since we're using the JSONOutputParser.

import asyncio
from gllm_inference.prompt_builder import PromptBuilder
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_inference.output_parser import JSONOutputParser
from gllm_inference.request_processor import LMRequestProcessor

prompt_builder = PromptBuilder(
    system_template="Answer in JSON format.", 
    user_template="What is the capital city of Indonesia?",
)
lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO)
output_parser = JSONOutputParser()
lm_request_processor = LMRequestProcessor(prompt_builder, lm_invoker, output_parser)
response = asyncio.run(lm_request_processor.process())
print(f"Response: {response}")

Expected Output

Response: {'capital': 'Jakarta'}

Using Prompt Variables

The LMRP also supports passing prompt variables to the prompt builder. Let's try it out!

import asyncio
from gllm_inference.prompt_builder import PromptBuilder
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_inference.request_processor import LMRequestProcessor

prompt_builder = PromptBuilder(system_template="Talk like a {role}.", user_template="{query}")
lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO)
lm_request_processor = LMRequestProcessor(prompt_builder, lm_invoker)
prompt_kwargs = {"role": "pirate", "query": "What is the capital city of Indonesia?"}
response = asyncio.run(lm_request_processor.process(prompt_kwargs=prompt_kwargs))
print(f"Response: {response}")

Expected Output

Response: Arrr, the cap’n o' Indonesia be Jakarta.

Adding History

The LMRP also supports passing history to the prompt builder. Let's try it out!

import asyncio
from gllm_inference.prompt_builder import PromptBuilder
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_inference.request_processor import LMRequestProcessor
from gllm_inference.schema import Message

history = [
    Message.user("What is the capital city of Indonesia?"),
    Message.assistant("Jakarta is the capital city of Indonesia."),
]
prompt_builder = PromptBuilder(user_template="In what island is it located?")
lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO)
lm_request_processor = LMRequestProcessor(prompt_builder, lm_invoker)
response = asyncio.run(lm_request_processor.process(history=history))
print(f"Response: {response}")

Expected Output

Response: Java.

Adding Extra Contents

The LMRP also supports passing extra contents to the prompt builder. Let's try it out!

import asyncio
from gllm_inference.prompt_builder import PromptBuilder
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_inference.request_processor import LMRequestProcessor
from gllm_inference.schema import Attachment

attachment = Attachment.from_path("path/to/tiger.jpg")
prompt_builder = PromptBuilder(user_template="What image is this?")
lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO)
lm_request_processor = LMRequestProcessor(prompt_builder, lm_invoker)
response = asyncio.run(lm_request_processor.process(extra_contents=[attachment]))
print(f"Response: {response}")

Expected Output

Response: A tiger (the striped big cat).

Automatic Tool Execution

When tools are provided to the language model, the LMRP has the capability to automatically executes the tools until the desired final response is produced. Let's test it out!

import asyncio
from gllm_inference.prompt_builder import PromptBuilder
from gllm_inference.lm_invoker import OpenAILMInvoker
from gllm_inference.model import OpenAILM
from gllm_inference.request_processor import LMRequestProcessor
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get the weather of a city."""
    return f"The weather of {city} is sunny."

prompt_builder = PromptBuilder(user_template="What is the weather in Jakarta?")
lm_invoker = OpenAILMInvoker(OpenAILM.GPT_5_NANO, tools=[get_weather])
lm_request_processor = LMRequestProcessor(prompt_builder, lm_invoker)
response = asyncio.run(lm_request_processor.process())
print(f"Response: {response}")

Expected Output

Response: Jakarta is sunny right now.

In the case that we don't want the LMRP to automatically executes the tool, we can set the auto_execute_tools param to False. In this case, the LMRP will directly return the ToolCall objects produced the language models. Let's try this by changing the following line from the above example:

...
response = asyncio.run(lm_request_processor.process(auto_execute_tools=False))
...

Expected Output

Response: LMOutput(
    tool_calls=[
        ToolCall(
            id='call_ufT7Ym9IPlWTN66TVcKnFJLA', 
            name='get_weather', 
            args={'city': 'Jakarta'},
        )
    ]
)

Use LM Request Processor Builder (Recommended)

This is a simpler way to initialize an LMRP, in which you can pass the essential parameters directly without manually creating individual components like prompt builders and LM invokers.

The build_lm_request_processor() function is a convenience method that automatically creates and configures all the necessary components for you:

from gllm_inference.builder import build_lm_request_processor
import os

# Simple LMRP creation with essential parameters
lm_request_processor = build_lm_request_processor(
    model_id="openai/gpt-4o-mini",
    credentials=os.getenv("OPENAI_API_KEY"),
    system_template="You are a helpful assistant that provides accurate information.",
    user_template="Please answer this question: {question}",
)

This single function call automatically:

1. Creates a PromptBuilder with your templates

2. Sets up the appropriate LMInvoker for your model

3. Combines them into a complete LMRequestProcessor

---

Congratulations! We've successfully completed the tutorial to use the LMRP!