To create a Databricks connection, you'll need an access token and the server hostname and path.
- Generate a personal access token in Databricks.
- Copy the generated token and store in a secure location.
Do one of the following:
If you're using a SQL endpoint:
- Click SQL Warehouses in the left nav.
- Choose an endpoint to connect to.
- Navigate to the Connection Details tab.
- Copy the Server Hostname and the HTTP Path.\
If you're using a compute cluster:
- Click Compute in the left nav.
- Choose a cluster to connect to.
- Navigate to Advanced Options.
- Click on the JDBC/ODBC tab.
- Copy the Server Hostname and the HTTP Path.
- Workspace URL - The Server Hostname of the compute cluster
- HTTP Path - The HTTP path for the compute cluster or the SQL Warehouse
- Token - The Databricks personal access token
- Catalog (Optional) - If you want to specify a Databricks Unity catalog, enter the catalog name here. You may leave this blank, in which case Lightup will connect to the default Databricks catalog, hive_metastore
- Schema scan frequency - Set how often scans run for the datasource: Hourly, Daily, or Weekly.
Databricks datasources support the Query History, Scheduling, Enable data storage, and Maximum backfill duration, and Maximum distinct values settings. For steps, see Set query governance settings for a datasource.
These Databricks date/time data types are supported:
These Databricks object types are supported:
Databricks datasources support partitions, multi-partitions, and partition time zones.
Updated 10 days ago