Database connectors

One interface across database families#

Igloo's goal is a common SQL and agent interface across databases, warehouses and lakehouses. Coverage grows through native adapters, compatible ADBC drivers, open-format snapshots, and planned replication or bridge integrations. All routes share DataFusion catalogs and the same MCP/JSON query contract.

This preview adds a generic ADBC adapter and named local Parquet sources with inferred schemas. PostgreSQL and SQLite ADBC 1.7.0 have live integration coverage. Other drivers are validation candidates; an installed driver does not establish certified Igloo compatibility.

Three implemented routes#

Choose a route that matches the source. Native PostgreSQL remains available alongside ADBC and named local Parquet catalogs.

Database coverage and evidence#

Verification currently covers Linux x86_64, SQLite/PostgreSQL ADBC 1.7.0 and PostgreSQL 15. The tests exercise metadata, allowlists, NULLs, quoted identifiers, projections, counts and joins, including native PostgreSQL joined to its ADBC route and SQLite joined to a distinct Parquet schema.

Configure a compatible ADBC driver#

Install a trusted driver outside Igloo and supply its absolute shared-library path. The current Rust manager loads the ADBC 1.1.0 C ABI. dbc can manage installation, but Igloo does not automatically resolve dbc manifests; use the driver's setup guide to locate its library.

One source represents one remote database/catalog. Metadata uses GetObjects and GetTableSchema. Set catalog explicitly if discovery returns multiple catalogs. Schema/table allowlists are exact and rechecked locally; a missing explicit table fails startup. Base tables are registered as source.schema.table; an absent remote schema uses public locally. ADBC does not add short aliases.

[[adbc_sources]]
name = "warehouse"
driver = "/absolute/path/to/adbc_driver.so"
# Or set IGLOO_SOURCE_WAREHOUSE_DRIVER.
# Supply IGLOO_SOURCE_WAREHOUSE_URI when the driver uses a URI.
catalog = "analytics"
schemas = ["public"]
tables = ["customers", "orders"]
dialect = "ansi" # ansi, mysql, mssql: identifier quoting only

[adbc_sources.options_env]
username = "WAREHOUSE_USER"
password = "WAREHOUSE_PASSWORD"

A SQLite source without PostgreSQL#

Install SQLite ADBC 1.7.0 through its trusted distribution, then set IGLOO_SOURCE_LOCAL_DB_DRIVER to the native library's absolute path. For example, the Linux adbc-driver-sqlite Python wheel includes libadbc_driver_sqlite.so. Python distributes the library; the Rust adapter does not require Python at query time.

Use an existing SQLite file with a customers table and save this TOML. No PostgreSQL source, dummy Parquet directory, CDC path or model key is required. Run igloo schema, then query local_db.public.customers or connect the same configuration through igloo mcp.

[[adbc_sources]]
name = "local_db"
uri = "/absolute/path/to/demo.sqlite"
tables = ["customers"]
# Set IGLOO_SOURCE_LOCAL_DB_DRIVER in the environment.

Named Parquet snapshots with inferred schemas#

Each named Parquet source is a catalog with tables in its public schema. Point tables at local Parquet files or directories. Schemas are inferred; customer/order schemas can be joined with native or ADBC data without editing Rust.

These are consistent exported snapshots. The optional legacy parquet_path still registers the old fixed-schema iceberg table, but it is not needed for named sources. S3/object-store support is not added by this local connector.

[[parquet_sources]]
name = "lake"
tables = [
  { name = "orders", path = "/data/snapshot/orders.parquet" },
  { name = "customers", path = "/data/snapshot/customers/" }
]
# Query lake.public.orders and lake.public.customers.

Live querying and replication are separate paths#

Columnar/dbc illustrates the standards-based driver route for database access. OLake illustrates source replication into Iceberg and Parquet. Igloo can build on compatible drivers for live queries, while a real lakehouse reader would let it query correctly replicated data from additional sources.

ADBC is not a CDC protocol. Igloo does not yet implement transaction-log replication, checkpoints, deletes, schema-change recovery or an OLake integration. Real Iceberg catalog/snapshot reads and interoperability tests are next work. A deliberately exported consistent local Parquet snapshot is usable today.

• Columnar database access
• dbc driver documentation
• OLake replication and open formats
• ADBC specification

Discover connector coverage from an agent#

Run igloo connectors for offline JSON coverage, or call igloo_connectors through MCP. This distinguishes tested integrations, driver candidates and planned routes. Use igloo_schema to inspect the tables actually registered by a configured instance, including their connector type.

Generic ADBC leaves filters and joins in DataFusion and uses database-specific quoting only for generated SELECT projections. Scans stream Arrow batches through a two-batch queue on blocking workers. This does not bound a driver's batch size or remote work, and dropping a timed-out query cannot interrupt a blocked C call. Configure source-side timeouts. Source pooling, coordinated snapshots and automatic schema refresh remain future work.

./target/debug/igloo connectors
./target/debug/igloo schema
./target/debug/igloo query --sql 'SELECT * FROM local_db.public.customers ORDER BY id'