# Supported Datastores

## Overview

The GLLM DataStore library implements the **Single Abstraction Pattern**, meaning you interact with a unified API regardless of the underlying storage engine. Whether you use Chroma for prototyping or Elasticsearch for production, the code to store and retrieve data remains generic:

```python
# The only difference is the initialization
store = ChromaDataStore(...)
# OR
store = ElasticsearchDataStore(...)

# The rest of your code stays the same
await store.vector.retrieve(query="...")
```

## Feature Matrix

| Datastore         | Fulltext | Vector | Hybrid | Notes                                          |
| ----------------- | -------- | ------ | ------ | ---------------------------------------------- |
| **Chroma**        | ✅        | ✅      | ❌      | Great for local dev & prototyping.             |
| **Elasticsearch** | ✅        | ✅      | ✅      | Production-grade search engine.                |
| **OpenSearch**    | ✅        | ✅      | ✅      | Open-source alternative to ES.                 |
| **Milvus**        | ✅        | ✅      | ✅      | High-performance vector database.              |
| **PostgreSQL**    | ✅        | ✅      | ❌      | Uses `pgvector` for vector support.            |
| **Redis**         | ✅        | ✅      | ❌      | In-memory speed with persistence.              |
| **SQL**           | ✅        | ❌      | ❌      | Generic SQL support (Postgres, MySQL, SQLite). |
| **In-Memory**     | ✅        | ✅      | ❌      | For testing and ephemeral data.                |

## Initialization Examples

To inspect the specific prerequisites for each datastore (e.g. `pip install ...`), please refer to the [Optional Dependencies](https://github.com/GDP-ADMIN/gl-sdk/blob/docs/gitbook-sync/gitbook/gen-ai-sdk/gen-ai-sdk/installation.md#optional-dependencies) section.

{% tabs %}
{% tab title="Chroma" %}

```python
from gllm_datastore.data_store import ChromaDataStore
from gllm_datastore.data_store.chroma.data_store import ChromaClientType
from gllm_inference.em_invoker import OpenAIEMInvoker

# 1. Initialize Invoker (for vector search)
em_invoker = OpenAIEMInvoker(model_name="text-embedding-3-small")

# 2. Initialize Store
store = (
    ChromaDataStore(
        collection_name="my_collection",
        client_type=ChromaClientType.PERSISTENT,
        persist_directory="./chroma-data",
    )
    .with_fulltext()
    .with_vector(em_invoker=em_invoker)
)
```

{% endtab %}

{% tab title="Elasticsearch" %}

```python
from gllm_datastore.data_store import ElasticsearchDataStore
from gllm_inference.em_invoker import OpenAIEMInvoker

em_invoker = OpenAIEMInvoker(model_name="text-embedding-3-small")

store = (
    ElasticsearchDataStore(
        index_name="my_index",
        url="http://localhost:9200",
        # api_key="..." or username="user", password="pass"
    )
    .with_fulltext()
    .with_vector(em_invoker=em_invoker)
)
```

{% endtab %}

{% tab title="OpenSearch" %}

```python
from gllm_datastore.data_store import OpenSearchDataStore
from gllm_inference.em_invoker import OpenAIEMInvoker

em_invoker = OpenAIEMInvoker(model_name="text-embedding-3-small")

store = (
    OpenSearchDataStore(
        index_name="my_index",
        url="http://localhost:9200",
        # username="user", password="pass"
    )
    .with_fulltext()
    .with_vector(em_invoker=em_invoker)
)
```

{% endtab %}

{% tab title="Milvus" %}

```python
from gllm_datastore.data_store import MilvusDataStore
from gllm_inference.em_invoker import OpenAIEMInvoker

em_invoker = OpenAIEMInvoker(model_name="text-embedding-3-small")

store = (
    MilvusDataStore(
        collection_name="my_collection",
        uri="http://localhost:19530",
        # token="root:Milvus", # Optional
    )
    .with_fulltext()
    .with_vector(em_invoker=em_invoker, dimension=1536)
)
```

{% endtab %}

{% tab title="PostgreSQL" %}

```python
from gllm_datastore.data_store.postgresql.data_store import PostgreSQLDataStore
from gllm_inference.em_invoker import OpenAIEMInvoker

em_invoker = OpenAIEMInvoker(model_name="text-embedding-3-small")

store = (
    PostgreSQLDataStore(
        engine_or_url="postgresql+asyncpg://user:password@localhost/dbname",
        table_name="chunks_table",
        embedding_dimension=1536
    )
    .with_fulltext()
    .with_vector(em_invoker=em_invoker)
)
```

{% endtab %}

{% tab title="Redis" %}

```python
from gllm_datastore.data_store import RedisDataStore
from gllm_inference.em_invoker import OpenAIEMInvoker

em_invoker = OpenAIEMInvoker(model_name="text-embedding-3-small")

store = (
    RedisDataStore(
        index_name="my_index",
        url="redis://localhost:6379",
    )
    .with_fulltext()
    .with_vector(em_invoker=em_invoker)
)
```

{% endtab %}

{% tab title="SQL" %}

```python
from gllm_datastore.data_store.sql.data_store import SQLDataStore

# Supports SQLite, MySQL, PostgreSQL (without vector)
store = (
    SQLDataStore(
        engine_or_url="sqlite+aiosqlite:///./my_database.db",
        table_name="my_text_chunks"
    )
    .with_fulltext()
)
```

{% endtab %}

{% tab title="In-Memory" %}

```python
from gllm_datastore.data_store import InMemoryDataStore
from gllm_inference.em_invoker import OpenAIEMInvoker

em_invoker = OpenAIEMInvoker(model_name="text-embedding-3-small")

store = (
    InMemoryDataStore()
    .with_fulltext()
    .with_vector(em_invoker=em_invoker)
)
```

{% endtab %}
{% endtabs %}
