Audio

Audio Loader is designed for extracting information from Audio file.

Prerequisites

This example specifically requires completion of all setup steps listed on the Prerequisites page.

Installation

Linux, macOS, or Windows WSL

# you can use a Conda environment
pip install --extra-index-url https://oauth2accesstoken:$(gcloud auth print-access-token)@glsdk.gdplabs.id/gen-ai-internal/simple/ "gllm-docproc[audio]"

Windows Powershell

Windows Command Prompt

You can use the following as a sample file: sample-audio-1.mp3.

Audio Loader

AudioLoader is responsible to extract information from audio file by utilizing the GLLM Multimodal Audio to Text.

1

Create a script called main.py:

from gllm_docproc.loader.audio import AudioLoader

source = "https://assets.analytics.glair.ai/generative/audio/sample-audio-1.mp3"

# initialize audio loader
loader = AudioLoader()

# load audio file
loaded_elements = loader.load(source)

2

Run the script:

python main.py

3

The loader will generate the following: output JSON.

Other Audio To Text

You can use other audio-to-text to customize the implementation. In this example, we use ProsaAudioToText from GLLM Multimodal ProsaAudioToText.

1

Create a script called main.py:

from gllm_docproc.loader.audio import AudioLoader
from gllm_multimodal.modality_converter.audio_to_text import ProsaAudioToText

source = "https://assets.analytics.glair.ai/generative/audio/sample-audio-1.mp3"

# initialize other audio to text (in this case using ProsaAudioToText)
prosa_audio_to_text = ProsaAudioToText(api_key="...", url="...", model="...")

# initialize audio loader
loader = AudioLoader([prosa_audio_to_text])

# load audio file
loaded_elements = loader.load(source)

2

Run the script:

python main.py

3

The loader will hit GL Speech service to convert the audio file to text.

GLLM Multimodal has the following implementations that you can use:

1. GeminiAudioToText

2. GoogleCloudAudioToText

3. OpenAIWhisperAudioToText

4. ProsaAudioToText

5. YouTubeTranscriptAudioToText

Custom Audio To Text

You can use fully custom audio-to-text to customize the implementation.

1

Create a script called custom_audio_to_text.py:

from gllm_multimodal.modality_converter.audio_to_text.audio_to_text import BaseAudioToText
from gllm_multimodal.modality_converter.schema.audio_transcript import AudioTranscript

class CustomAudioToText(BaseAudioToText):
  def __init__(self, ...):
    # your custom initialization

  async def convert(self, audio_source: str) -> list[AudioTranscript]:
    # your custom implementation

2

Create a script called main.py:

import CustomAudioToText
from gllm_docproc.loader.audio import AudioLoader

source = "https://assets.analytics.glair.ai/generative/audio/sample-audio-1.mp3"

# initialize custom audio to text (in this case using your own custom implementation)
custom_audio_to_text = CustomAudioToText(...)

# initialize audio loader
loader = AudioLoader([custom_audio_to_text])

# load audio file
loaded_elements = loader.load(source)

3

Run the script:

python main.py

4

The loader will use your custom implementation to convert the audio file to text.