CSV

gllm-docproc | Tutorial: CSV Loader | Use Case: Advanced DPO Pipeline | API Reference

CSV Loader is a component designed for extracting information from CSV documents and converting it into a standardized JSON format.

Prerequisites

This example specifically requires completion of all setup steps listed on the Prerequisites page.

Installation

Linux, macOS, or Windows WSL

# you can use a Conda environment
pip install --extra-index-url https://oauth2accesstoken:$(gcloud auth print-access-token)@glsdk.gdplabs.id/gen-ai-internal/simple/ "gllm-docproc[csv]"

Windows Powershell

Windows Command Prompt

You can use the following as a sample file: customers-1000.csv.

Pandas Loader

PandasLoader is responsible to extract tabular data from CSV and similar delimited text formats using the pandas library.

1

Create a script called main.py:

from gllm_docproc.loader.csv import PandasLoader

source = "./data/source/customers-1000.csv"

# initialize the PandasLoader
loader = PandasLoader()

# load the csv source
loaded_elements = loader.load(source)

2

Run the script:

python main.py

3

The loader will generate the following: output JSON.