Adding Document References

This guide will walk you through adding a Reference Formatter component to your existing RAG pipeline that automatically formats and includes source references in your responses, making your answers more credible and traceable.

Reference formatting enhances your RAG responses with automatic source citations, providing transparency and credibility by showing exactly where information came from

This tutorial extends the Your First RAG Pipeline tutorial. Ensure you have followed the instructions to set up your repository.

Prerequisites

This example specifically requires:

1. Completion of the Your First RAG Pipeline tutorial - this builds directly on top of it

2. Completion of all setup steps listed on the Prerequisites page

3. A working OpenAI API key configured in your environment variables

You should be familiar with these concepts and components:

1. Components in Your First RAG Pipeline - Required foundation

2. Reference Formatter

View full project code on GitHub

Installation

Linux, macOS, or Windows WSL

# you can use a Conda environment
pip install --extra-index-url "https://oauth2accesstoken:$(gcloud auth print-access-token)@glsdk.gdplabs.id/gen-ai-internal/simple/" gllm-rag gllm-core gllm-generation gllm-inference gllm-pipeline gllm-retrieval gllm-misc gllm-datastore

Windows Powershell

Windows Command Prompt

---

You can either:

1. You can refer to the guide whenever you need explanation or want to clarify how each part works.

2. Follow along with each step to recreate the files yourself while learning about the components and how to integrate them.

Both options will work—choose based on whether you prefer speed or learning by doing!

Project Setup

1

Extend Your RAG Pipeline Project

Start with your completed RAG pipeline project from the Your First RAG Pipeline tutorial. We don't need to add any new file for this tutorial. Therefore, the structure should stay as is:

<project-name>/
├── data/
│   ├── <index>/...                     # preset data index folder
│   ├── chroma.sqlite3                  # preset database file
│   ├── imaginary_animals.csv           # sample data
├── modules/
│   ├── retriever.py
│   └── response_synthesizer.py
├── .env
├── indexer.py                    
└── pipeline.py    # 👈 Will be updated with reference formatter

---

1) Build the Reference Formatter Pipeline

The SimilarityBasedReferenceFormatter analyzes your response against retrieved chunks and automatically creates formatted citations using chunk metadata.

1

Create the reference formatter step

Update your pipeline.py (or create a new one) with the reference formatter:

from gllm_generation.reference_formatter import SimilarityBasedReferenceFormatter
from gllm_pipeline.steps import step

reference_formatter = SimilarityBasedReferenceFormatter(
    em_invoker=em_invoker, threshold=0.5, stringify=False
)

format_reference_step = step(
    component=reference_formatter,
    input_map={"response": "response", "chunks": "chunks"},
    output_state="references",
)

2

Compose the final pipeline

Chain all steps including the reference formatter:

e2e_pipeline = retrieve_step | synthesize_step | format_reference_step

This creates a pipeline that generates responses with automatic source citations from the retrieved chunks.

🧠 The RAGState input state already contains a reference field that gets populated by the reference formatter.

2) Run the Pipeline

When running the pipeline, you may encounter an error like this:

[2025-08-26T14:36:10+0700.550 chromadb.telemetry.product.posthog ERROR] Failed to send telemetry event CollectionQueryEvent: capture() takes 1 positional argument but 3 were given

Don't worry about this, since we do not use this Chroma feature. Your Pipeline should still work.

1

Configure and invoke the pipeline

Configure the state and config for direct pipeline invocation:

async def main():
    state = {"user_query": "Give me nocturnal creatures from the dataset"}  # Replace with your actual query
    config = {"top_k": 5}
    result = await e2e_pipeline.invoke(state, config)
    print(f"Pipeline result: {result['response']}")
    print(f"References: {result['references']}")


if __name__ == "__main__":
    asyncio.run(main())

2

Observe output

If you successfully run all the steps, you will see something like this appended in the end of the result:

References: [Chunk(id=ffe9bfbf-6065-480c-bded-276ac9994d72, content=The Luminafox is a nocturnal creature inhabiting t..., metadata={'name': 'Luminafox'}, score=0.4633599574686413), Chunk(id=b91d6dfb-f177-48c9-84dd-a782b785ee0d, content=The Dusk Panther prowls the twilight forests of Sh..., metadata={'name': 'Dusk Panther'}, score=0.4541605560513348), Chunk(id=05429706-c477-4804-a4e5-9da1ddf50632, content=The Gloombat flits through the dark caverns of Dus..., metadata={'name': 'Gloombat'}, score=0.4435190881711845), Chunk(id=4965e21b-4780-4a76-9d6a-b70e4faf1da2, content=The Moonstalker is a nocturnal predator prowling t..., metadata={'name': 'Moonstalker'}, score=0.44225796877897344), Chunk(id=d95382d7-2fdb-41cf-b741-ec0ceaed732d, content=The Glowhopper is an insect-like creature residing..., metadata={'name': 'Glowhopper'}, score=0.42312825178877955)]

Troubleshooting

Common Issues

1. No references being generated:

   • Ensure chunks have meaningful metadata

   • Check that the reference formatter step is included in the pipeline

   • Verify the response contains content that matches chunk content

2. Poor reference quality:

   • Improve chunk metadata with more descriptive information

   • Ensure chunks are properly indexed with relevant content

   • Check that the response synthesizer generates content that references the chunks

3. Reference formatting issues:

   • Verify the SimilarityBasedReferenceFormatter is properly configured

   • Check that chunk metadata is in the expected format

   • Ensure the pipeline state includes the reference field

Debug Tips

1. Enable debug mode: Set debug: true in your request to see detailed logs

2. Check chunk metadata: Verify that your chunks have meaningful metadata

3. Examine response content: Ensure the response actually references the chunk content

4. Pipeline step order: Confirm the reference formatter step comes after response generation

---

Congratulations! You've successfully implemented a Reference Formatter component in your RAG pipeline. This enhancement makes your responses more credible and traceable by automatically including formatted references to the source information, significantly improving the transparency and reliability of your AI-powered answers.