JSON

gllm-docproc | Tutorial: JSON Loader | Use Case: Advanced DPO Pipeline | API Reference

JSON Loader is a component designed for extracting information from a JSON file and converting it into a standardized JSON format.

Prerequisites

This example specifically requires completion of all setup steps listed on the Prerequisites page.

Installation

Linux, macOS, or Windows WSL

# you can use a Conda environment
pip install --extra-index-url https://oauth2accesstoken:$(gcloud auth print-access-token)@glsdk.gdplabs.id/gen-ai-internal/simple/ "gllm-docproc"

Windows Powershell

Windows Command Prompt

You can use the following as a sample file: json-elements-example.json.

JSON Elements Loader

JSONElementsLoader is responsible to extract information from JSON file that contain a list of dictionaries following the Element model structure.

1

Create a script called main.py:

from gllm_docproc.loader.json import JSONElementsLoader

source = "./data/source/json-elements-example.json"

# initialize the JSONElementsLoader
loader = JSONElementsLoader()

# load the json elements source
loaded_elements = loader.load(source)

2

Run the script:

python main.py

3

The loader will generate the following: output JSON.