Dynamic Chunking

Get File Chunks

get

Endpoint to get chunks by file id from a specified knowledge base with pagination.

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

file_idstringRequired

The ID of the file

Query parameters

pageintegerOptional

Page number (0-based)

Default: 0

sizeinteger · min: 1 · max: 1000Optional

Number of chunks per page

Default: 20

Responses

200

Successful Response

application/json

404

Not Found

application/json

422

Validation Error

application/json

500

Internal Server Error

application/json

get

/{knowledge_base_id}/files/{file_id}/chunks

HTTPcURLJavaScriptPython

GET /{knowledge_base_id}/files/{file_id}/chunks HTTP/1.1
Accept: */*

{
  "chunks": [
    {
      "text": "The text content of the chunk",
      "structure": "uncategorized",
      "metadata": {
        "source": "pdf-example.pdf",
        "source_type": "pdf",
        "loaded_datetime": "2025-04-29 00:00:00",
        "file_id": "12345",
        "chunk_id": "12345-5",
        "previous_chunk": "12345-4",
        "next_chunk": "12345-6",
        "order": 5
      }
    }
  ],
  "pagination": {
    "page": 4,
    "size": 1,
    "total_chunks": 10,
    "total_pages": 10,
    "has_next": true,
    "has_previous": true
  }
}

Create Chunk

post

Endpoint to create a new chunk based on the file id to a specified knowledge base.

This endpoint will:

1. Generate chunk metadata: chunk ID, relationships with adjacent chunks.

2. Process the chunk through the Data Generator and Indexer.

3. Update the metadata of neighboring chunks to maintain proper relationships.

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

file_idstringRequired

The ID of the file

Body

application/json

The body of the request

next_chunkany ofRequired

The ID of the chunk that should follow the created chunk in the sequence.

stringOptional

or

nullOptional

previous_chunkany ofRequired

The ID of the chunk that should precede the created chunk in the sequence.

stringOptional

or

nullOptional

Responses

200

Successful Response

application/json

400

Bad Request

application/json

422

Validation Error

application/json

500

Internal Server Error

application/json

post

/{knowledge_base_id}/files/{file_id}/chunks

HTTPcURLJavaScriptPython

POST /{knowledge_base_id}/files/{file_id}/chunks HTTP/1.1
Content-Type: application/json
Accept: */*
Content-Length: 246

{
  "next_chunk": "12345-2",
  "previous_chunk": "12345-1",
  "chunk": {
    "text": "The text content of the chunk",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345"
    }
  }
}

{
  "created_chunk": {
    "text": "chunk text",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-1.5",
      "previous_chunk": "12345-1",
      "next_chunk": "12345-2",
      "order": 1.5
    }
  },
  "updated_next_chunk": {
    "text": "next chunk text",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-2",
      "previous_chunk": "12345-1.5",
      "next_chunk": "12345-3",
      "order": 2
    }
  },
  "updated_previous_chunk": {
    "text": "previous chunk text",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-1",
      "next_chunk": "12345-1.5",
      "order": 1
    }
  }
}

Delete File Chunks

delete

Endpoint to delete all file chunks by file id from a knowledge base.

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

file_idstringRequired

The ID of the file

Responses

200

Successful Response

application/json

404

Not Found

application/json

422

Validation Error

application/json

500

Internal Server Error

application/json

delete

/{knowledge_base_id}/files/{file_id}/chunks

HTTPcURLJavaScriptPython

DELETE /{knowledge_base_id}/files/{file_id}/chunks HTTP/1.1
Accept: */*

{
  "file_id": "1234"
}

Get Chunk

get

Endpoint to get a chunk based on the chunk id from a file in a specified knowledge base.

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

file_idstringRequired

The ID of the file

chunk_idstringRequired

The ID of the chunk

Responses

200

Successful Response

application/json

404

Not Found

application/json

422

Validation Error

application/json

500

Internal Server Error

application/json

get

/{knowledge_base_id}/files/{file_id}/chunks/{chunk_id}

HTTPcURLJavaScriptPython

GET /{knowledge_base_id}/files/{file_id}/chunks/{chunk_id} HTTP/1.1
Accept: */*

{
  "text": "The text content of the chunk",
  "structure": "uncategorized",
  "metadata": {
    "source": "pdf-example.pdf",
    "source_type": "pdf",
    "loaded_datetime": "2025-04-29 00:00:00",
    "file_id": "12345",
    "chunk_id": "12345-5",
    "previous_chunk": "12345-4",
    "next_chunk": "12345-6",
    "order": 5
  }
}

Update Chunk

put

Endpoint to update a chunk based on the chunk id from a file in a specified knowledge base.

This endpoint will:

1. Update the chunk's text content and/or metadata, but ignoring the system-managed metadata fields (file_id, chunk_id, previous_chunk, next_chunk, parent_chunk, children_chunk, order)

2. When updating text content:

• Triggers re-processing of the chunk using the data generator

• Re-indexes the chunk with updated vector embeddings based on the new text

1. When only updating metadata:

• Can only add or modify existing metadata fields

• Cannot remove existing metadata keys

• System-managed metadata fields (file_id, chunk_id, etc.) are preserved

1. Return the updated chunk information

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

file_idstringRequired

The ID of the file

chunk_idstringRequired

The ID of the chunk

Body

application/json

The body of the request

textstringRequired

The text content of the chunk.

structurestringOptional

The structure of the chunk.

Default: uncategorized

Responses

200

Successful Response

application/json

404

Not Found

application/json

422

Validation Error

application/json

500

Internal Server Error

application/json

put

/{knowledge_base_id}/files/{file_id}/chunks/{chunk_id}

HTTPcURLJavaScriptPython

PUT /{knowledge_base_id}/files/{file_id}/chunks/{chunk_id} HTTP/1.1
Content-Type: application/json
Accept: */*
Content-Length: 186

{
  "text": "The text content of the chunk",
  "structure": "uncategorized",
  "metadata": {
    "source": "pdf-example.pdf",
    "source_type": "pdf",
    "loaded_datetime": "2025-04-29 00:00:00",
    "file_id": "12345"
  }
}

{
  "updated_chunk": {
    "text": "The updated text content of the chunk",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-2",
      "previous_chunk": "12345-1",
      "next_chunk": "12345-3",
      "order": 2
    }
  }
}

Delete Chunk

delete

Endpoint to delete a chunk based on the chunk id from a file in a specified knowledge base.

This endpoint will:

1. Delete the chunk from the vector database.

2. Update the metadata of the next and previous chunks to reflect the deletion.

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

file_idstringRequired

The ID of the file

chunk_idstringRequired

The ID of the chunk

Responses

200

Successful Response

application/json

404

Not Found

application/json

422

Validation Error

application/json

500

Internal Server Error

application/json

delete

/{knowledge_base_id}/files/{file_id}/chunks/{chunk_id}

HTTPcURLJavaScriptPython

DELETE /{knowledge_base_id}/files/{file_id}/chunks/{chunk_id} HTTP/1.1
Accept: */*

{
  "deleted_chunk": {
    "text": "Chunk text",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-5",
      "previous_chunk": "12345-4",
      "next_chunk": "12345-6",
      "order": 5
    }
  },
  "updated_next_chunk": {
    "text": "next chunk text",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-6",
      "previous_chunk": "12345-4",
      "next_chunk": "12345-7",
      "order": 6
    }
  },
  "updated_previous_chunk": {
    "text": "previous chunk text",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-4",
      "previous_chunk": "12345-3",
      "next_chunk": "12345-6",
      "order": 4
    }
  }
}

Update Chunk Metadata

patch

Endpoint to update a chunk metadata based on the chunk id from a file in a specified knowledge base.

This endpoint will:

1. Update only the chunk's metadata fields, but ignoring the system-managed metadata fields (file_id, chunk_id, previous_chunk, next_chunk, parent_chunk, children_chunk, order)

2. When updating metadata:

• Can only add or modify existing metadata fields (source, source_type, loaded_datetime, etc.)

• Cannot remove existing metadata keys

1. Return the updated chunk information with the modified metadata

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

file_idstringRequired

The ID of the file

chunk_idstringRequired

The ID of the chunk

Body

application/json

The chunk metadata to update

Other propertiesanyOptional

Responses

200

Successful Response

application/json

404

Not Found

application/json

422

Validation Error

application/json

500

Internal Server Error

application/json

patch

/{knowledge_base_id}/files/{file_id}/chunks/{chunk_id}

HTTPcURLJavaScriptPython

PATCH /{knowledge_base_id}/files/{file_id}/chunks/{chunk_id} HTTP/1.1
Content-Type: application/json
Accept: */*
Content-Length: 88

{
  "source": "pdf-example.pdf",
  "source_type": "pdf",
  "loaded_datetime": "2025-04-29 00:00:00"
}

{
  "updated_chunk": {
    "text": "Chunk text with the updated metadata",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-2",
      "previous_chunk": "12345-1",
      "next_chunk": "12345-3",
      "order": 2
    }
  }
}

Merge Chunks

post

Endpoint to merge multiple chunks based on the chunk ids from the request body.

This endpoint will:

1. Merge the chunks (text and metadata) based on the chunk ids.

2. Process the merged chunk through DPO pipeline (from Data Generator component).

3. Delete the original chunks that are merged.

4. Update the metadata of next chunk to maintain proper relationships.

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

file_idstringRequired

The ID of the file

Body

application/json

The list of chunk IDs to merge

chunk_idsstring[]Required

The IDs of the chunks to merge.

Responses

200

Successful Response

application/json

400

Bad Request

application/json

404

Not Found

application/json

422

Validation Error

application/json

500

Internal Server Error

application/json

post

/{knowledge_base_id}/files/{file_id}/chunks/merge

HTTPcURLJavaScriptPython

POST /{knowledge_base_id}/files/{file_id}/chunks/merge HTTP/1.1
Content-Type: application/json
Accept: */*
Content-Length: 45

{
  "chunk_ids": [
    "12345-1",
    "12345-2",
    "12345-3"
  ]
}

{
  "merged_chunk": {
    "text": "The first chunk text\nThe second chunk text\nThe third chunk text",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-01-01 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-1",
      "next_chunk": "12345-4",
      "order": 1
    }
  },
  "updated_next_chunk": {
    "text": "The fourth chunk text",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-01-01 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-4",
      "previous_chunk": "12345-1",
      "next_chunk": "2345-5",
      "order": 4
    }
  }
}

Process File

post

Endpoint to process a file to a specified knowledge base.

⚠️ WARNING: This endpoint is not implemented yet.

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

Body

multipart/form-data

input_typestring · enumRequired

The type of the file. File type could be FILE or URL.

Possible values:

file_idany ofOptional

The ID of the file. If not provided, the file id will be generated.

stringOptional

or

nullOptional

fileany ofOptional

The file to be processed.

string · binaryOptional

or

nullOptional

urlany ofOptional

The url of the file to be processed.

stringOptional

or

nullOptional

Responses

200

Successful Response

application/json

Responseany

422

Validation Error

application/json

post

/{knowledge_base_id}/files

HTTPcURLJavaScriptPython

POST /{knowledge_base_id}/files HTTP/1.1
Content-Type: multipart/form-data
Accept: */*
Content-Length: 67

{
  "input_type": "FILE",
  "file_id": "text",
  "file": "binary",
  "url": "text"
}

No content

Process File to Chunks

post

This endpoint accept POST request to process a file to chunks with configuration specified in the request. If no configuration is specified, the default configuration in the knowledge base will be used. The input to be processed can be a file upload or a URL.

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

Body

multipart/form-data

inputany ofRequired

The input to be processed. Can be a file upload or a URL.

string · binaryOptional

or

stringOptional

input_typestring · enumRequired

The type of the input. Can be FILE or URL.

Possible values:

input_file_nameany ofOptional

The name of the input file. Required for non-UploadFile inputs when type is FILE.

stringOptional

or

nullOptional

processing_configany ofOptional

The DPO processing configuration.

stringOptional

or

nullOptional

Responses

200

Successful Response

application/json

400

Bad Request

application/json

422

Validation Error

application/json

500

Internal Server Error

application/json

post

/{knowledge_base_id}/chunks

HTTPcURLJavaScriptPython

POST /{knowledge_base_id}/chunks HTTP/1.1
Content-Type: multipart/form-data
Accept: */*
Content-Length: 326

{
  "input": "https://assets.analytics.glair.ai/generative/pdf/pdf-example.pdf",
  "input_type": "FILE",
  "input_file_name": "pdf-example.pdf",
  "processing_config": "{\"chunker\": {\"text_chunk_size\": 1000,\"text_separators\": \"[\"\\n#\", \"\\n\", \" \", \"\"]\",\"table_format\": \"html\",\"is_parent_structure_info_included\": true}}"
}

{
  "chunks": [
    {
      "text": "This is a text of chunk 1",
      "structure": "uncategorized",
      "metadata": {
        "source": "pdf-example.pdf",
        "source_type": "pdf",
        "loaded_datetime": "2025-04-29 00:00:00",
        "file_id": "12345",
        "chunk_id": "12345-1",
        "next_chunk": "12345-2"
      }
    },
    {
      "text": "This is a text of chunk 2",
      "structure": "uncategorized",
      "metadata": {
        "source": "pdf-example.pdf",
        "source_type": "pdf",
        "loaded_datetime": "2025-04-29 00:00:00",
        "file_id": "12345",
        "chunk_id": "12345-2",
        "previous_chunk": "12345-1"
      }
    }
  ]
}

Split Chunk

post

Endpoint to split a chunk based on the chunk id, and the split indices from the request body.

This endpoint will:

1. Split the existing chunk into two new chunks at the specified split index.

2. Maintain proper ordering and relationships between chunks.

3. Process the new chunks through the Data Generator and Indexer.

4. Update the metadata of next chunk to maintain proper relationships.

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

file_idstringRequired

The ID of the file

chunk_idstringRequired

The ID of the chunk

Body

application/json

The split indices

split_indexintegerRequired

The index of the split point. Valid range: 1 to length-1

Responses

200

Successful Response

application/json

400

Bad Request

application/json

404

Not Found

application/json

422

Validation Error

application/json

500

Internal Server Error

application/json

post

/{knowledge_base_id}/files/{file_id}/chunks/{chunk_id}/split

HTTPcURLJavaScriptPython

POST /{knowledge_base_id}/files/{file_id}/chunks/{chunk_id}/split HTTP/1.1
Content-Type: application/json
Accept: */*
Content-Length: 19

{
  "split_index": 100
}

{
  "first_chunk": {
    "text": "First part of the chunk",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-1",
      "next_chunk": "12345-1.5",
      "order": 1
    }
  },
  "second_chunk": {
    "text": "Second part of the chunk",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-1.5",
      "previous_chunk": "12345-1",
      "next_chunk": "12345-2",
      "order": 1.5
    }
  },
  "updated_next_chunk": {
    "text": "Next chunk",
    "structure": "uncategorized",
    "metadata": {
      "source": "pdf-example.pdf",
      "source_type": "pdf",
      "loaded_datetime": "2025-04-29 00:00:00",
      "file_id": "12345",
      "chunk_id": "12345-2",
      "previous_chunk": "12345-1.5",
      "order": 2
    }
  }
}

Get Processing Configuration

get

Endpoint to get the processing configuration for specified knowledge base.

Path parameters

knowledge_base_idstringRequired

The ID of the knowledge base

Responses

200

Successful Response

application/json

Responseany

422

Validation Error

application/json

500

Internal Server Error

application/json

get

/{knowledge_base_id}/processing-config

HTTPcURLJavaScriptPython

GET /{knowledge_base_id}/processing-config HTTP/1.1
Accept: */*

{
  "type": "object",
  "title": "Document Processing Configuration",
  "description": "Complete configuration for document processing pipeline",
  "properties": {
    "chunker": {
      "type": "object",
      "title": "Chunker Configuration",
      "description": "Configure how documents should be chunked",
      "properties": {
        "text_chunk_size": {
          "type": "integer",
          "title": "Text Chunk Size",
          "description": "The maximum number of characters in each text chunk",
          "default": 1800
        },
        "text_chunk_overlap": {
          "type": "integer",
          "title": "Text Chunk Overlap",
          "description": "The number of overlapping characters between consecutive text chunks",
          "default": 360
        },
        "text_separators": {
          "type": "string",
          "title": "Text Separators",
          "description": "List of separators used to split text into chunks, in order of preference",
          "default": "[\"\\n#\", \"\\n\\n\", \"\\n\", \". \", \"! \", \"? \", \", \", \" \", \"\"]"
        },
        "table_chunk_size": {
          "type": "integer",
          "title": "Table Chunk Size",
          "description": "The maximum number of cells in each table chunk",
          "default": 4000
        },
        "table_chunk_overlap": {
          "type": "integer",
          "title": "Table Chunk Overlap",
          "description": "The number of overlapping cells between consecutive table chunks",
          "default": 0
        },
        "table_format": {
          "type": "string",
          "title": "Table Format",
          "description": "The format for table chunks",
          "enum": [
            "csv",
            "html",
            "markdown"
          ],
          "default": "markdown"
        },
        "is_parent_structure_info_included": {
          "type": "boolean",
          "title": "Include Parent Structure Info",
          "description": "Whether to include parent structural information in each chunk",
          "default": true
        }
      },
      "additionalProperties": false
    }
  },
  "additionalProperties": false
}