SQL
Module name: sql
The GLConnectors SQL Connector provides a powerful and flexible way to connect to and query your SQL databases directly from GL Connectors' servers. By leveraging the robustness of SQLAlchemy, it supports a wide range of database systems, allowing you to seamlessly integrate your existing data infrastructure.
This guide will walk you through configuring and how to use the SQL Connector.
Configuration Parameters
You can configure a new database integration in two ways: by providing a full connection url or by specifying individual connection parameters.
When initiating the integration, you pass a configuration dictionary. If the url field is provided, all other connection-specific fields are ignored.
Required Fields
This is the simplest method. Provide the full database connection string.
Required Fields:
url: The full URL to the database, including the protocol and hostname.
config = {
"url": "postgresql://myuser:mypassword@myhost:5432/mydatabase",
"identifier": "My-Production-DB" # Optional, but highly recommended
}Use this method if you prefer to provide each connection detail separately.
Required Fields:
host: The hostname of the database.port: The port of the database.database: The name of the database.username: The username for authentication.password: The password for authentication.driver: The SQLAlchemy driver to use for the connection.
config = {
"host": "db.example.com",
"port": 3306,
"database": "analytics_db",
"username": "analyst",
"password": "secure_password_123",
"driver": "mysql+pymysql",
"identifier": "MySQL-Analytics" # Optional, but highly recommended
}Additional Optional Fields
identifier: A custom string to help you identify this specific integration later. While this field is optional, it is highly recommended. If not provided, the identifier defaults to a concatenated string of connection details (driver:host:port:username:database), which can become long and difficult to read. Setting a clear, human-readable identifier is best practice.extra_config: A dictionary for any extra parameters required by the database driver (e.g., SSL settings).
Drivers
If the connection fails because we currently do not support the database (see below), please contact GL Connectors Team. As long as there is a driver available that can be installed that supports that SQL database type, we are willing to provide the support as needed.
Drivers must follow SQLAlchemy's supported engine that can be found here. The following database types have been tested:
MySQL
mysql
MariaDB
mysql or mariadb
PostgreSQL
postgresql
AWS Athena
awsathena+rest
Microsoft SQL Server
mssql+pymssql
Usage Example
Connecting to Athena is slightly trickier. We need to use awsathena+rest for the driver name, and the rest of the configuration requires the AWS configuration data as follows:
Variable Descriptions:
{AWS_CLIENT_ID}: Your AWS Access Key ID.{AWS_CLIENT_SECRET}: Your AWS Secret Access Key.{AWS_REGION}: The AWS region where your Athena instance is running (e.g.,us-east-1).{ATHENA_DB_SOURCE}: The name of your Athena database or data source (oftenAwsDataCatalog).{AWS_ATHENA_S3}: The S3 bucket name used by Athena to store query results (e.g.,my-athena-query-results-bucket).{AWS_WORKGROUP}: The Athena workgroup you want to use (e.g.,primary).
You can also use the following configuration following the individual parameters guide:
Last updated