Direct File URL

Direct File URL Downloader allows you to download files from a direct file URL and save them to a specified output directory. A direct file URL is a URL that points directly to a downloadable file (e.g., https://assets.analytics.glair.ai/generative/pdf/pdf-example.pdf).

Prerequisites

If you want to try the snippet code in this page:

• Completion of all setup steps listed on the Prerequisites page.

Installation

Linux, macOS, or Windows WSL

# you can use a Conda environment
pip install --extra-index-url https://oauth2accesstoken:$(gcloud auth print-access-token)@glsdk.gdplabs.id/gen-ai-internal/simple/ "gllm-docproc"

Windows Powershell

Windows Command Prompt

Basic Usage

1

Create a script called main.py:

from gllm_docproc.downloader import DirectFileURLDownloader

source = "https://assets.analytics.glair.ai/generative/pdf/pdf-example.pdf"
output_path = "./downloader/output"

# Initialize downloader
downloader = DirectFileURLDownloader()

# Download file
downloaded_files = downloader.download(source, output_path)
print(f"Downloaded files: {downloaded_files}")

2

Run the script:

python main.py

3

The downloader will save the file to the specified output directory with an automatically generated filename based on timestamp and UUID. The file extension will be detected from the HTTP response headers or file content.