If you're new to Lightup this Quick Start will help you jump right in. # Get oriented in the UI The Lightup app has three main areas:

1. The [top bar](#the-top-bar-navigation) has a workspace switcher on the right to switch between workspaces. Workspace and Admin settings can be accessed by clicking on the configuration icon next to the user name. Other user actions are possible via clicking on the user icon (such as contact support, generate API credentials, and logout) 2. The [left pane](#the-left-pane-app-settings-and-workspaces) lists the menu of data quality related actions you will be interested in accessing for the workspace selected. 3. The main page changes to reflect your choices on the left pane and the top bar. This is where you work with data quality components inside a specific workspace. ## The top pane: Workspace switcher, Admin and workspace settings Your [app role](doc:roles#app-roles) establishes what you can do outside of workspaces. Workspaces also have [roles](roles#workspace-roles) that determine what you can do inside of them. Look under the username icon to see which app role you have: * If you're an App Admin, you'll notice an **Admin** menu right under the app logo. As an App Admin you can create and manage Lightup user accounts, roles, and workspaces.

* If you're an App Editor, you won't find an **Admin** setting, but you will notice the **Add New Workspace** in the workspace drop down. As an App Editor, you can create and manage workspaces, and can invite other Lightup app users to your workspaces. /im * If you don't have an **Admin** menu or an **Add New Workspace** button, you're an App Viewer. You can be invited to join a workspace. In the left pane, you'll see a list of workspaces where you're a member. Select the arrow at the right edge of a listed workspace to open a menu of commands for that workspace. ## The left pane: navigation The top bar offers a set of tabs you can select to focus on a specific quality monitoring process in a workspace. * Select **Explorer** to manage data quality while viewing your data asset structure in your [Explorer tree](doc:managing-your-explorer-tree), which shows all of the workspace's datasources. * Select **Metrics** to [focus on metrics](doc:create-and-edit-metrics) (the aggregated, periodic queries that measure data quality). * Select **Monitors** to [focus on monitors](monitor-a-metric) (the regular checking of metric values). * Select **Incidents** to [review recent data quality incidents](doc:manage-incidents). * Select **Dashboards** to open the workspace's [data quality](data-quality-dashboards) or [incident](doc:incident-dashboards) dashboards. ## The main page: get stuff done All data quality work you do in the Lightup app you do in the main page. When you pick a workspace its **Explorer** tab appears in the main page, displaying a tree view of your data assets on the left and asset-focused tabs with data quality analysis tools on the right. This combination lets you quickly set up your asset hierarchy for data quality analysis and health monitoring, from datasources down to individual columns. The following section walks you through the process of setting up basic data quality monitoring. # Start monitoring data quality **Note:** you'll need at least the App Editor app role to follow along with all the steps in this section. If you're an App Viewer, you may still be able follow along with most of the steps, depending on your [workspace role](roles#workspace-roles): * If you have the Workspace Admin role in a workspace, you can start by [connecting to a datasource](#connect-to-a-datasource). * Or, if you have the Workspace Editor in a workspace that has at least one datasource connection available, you can start by [adding schemas to Explorer](#add-schemas-to-explorer). Now that you know how Lightup is laid out, you may be wondering, "What do I do next?". Using Lightup is easy, but you have to start somewhere. The following series of examples walks you through the process of setting up a data quality indicator. Once you've learned that basic flow, you're on your way to implementing an always-on observability platform that's customized for your data and business use cases. ## Create a workspace The first step is creating the workspace where you'll set up your data quality objects— the data assets you will analyze, and the metrics and monitors you create to perform that analysis. 1. On the top pane, select **Create new workspace** from under the workspace switcher drop down.

2. On the **New Workspace** page, enter a **Workspace Name** and **Description**. 3. Select **Save** at the top right corner.

## Connect to a datasource To begin adding your data assets, you connect to a datasource— a source database. Note that the source database must be prepared before you can connect to it. These preparatory steps are provided and need only be performed once per source database. You must have the [Workspace Admin role](doc:roles#workspace-roles) to connect to a datasource, or to edit an existing datasource connection. After you connect to a datasource it becomes available to others for data quality analysis. A datasource exists within a workspace— it points to a source database. You can add a datasource connection in each workspace where you need it. This allows people working in different workspaces to independently analyze and monitor the same source databases. Each type of datasource has its own inputs required for connecting to it. For steps, see the datasource's page in the following list. * [Amazon S3 bucket (Beta)](doc:s3) * [Athena](connect-to-athena) * [Azure Blob Storage (Beta)](doc:azure-blob-storage) * [BigQuery](connect-to-bigquery) * [Databricks](connect-to-databricks) * [Dremio](doc:dremio) * [Greenplum](greenplum) * [Hive](connect-to-hive) * [Incorta](incorta) * [Microsoft SQL](connect-to-microsoft-sql) * [MySQL](doc:mysql) * [Oracle](connect-to-oracle) * [Postgres](connect-to-postgres) * [Redshift](connect-to-redshift) * [SAP HANA](https://docs.lightup.ai/docs/saphana) * [Snowflake](connect-to-snowflake) * [Teradata](https://docs.lightup.ai/docs/saphana) For example, here's how to connect to a new datasource using a Postgres database that's already prepared, with an example shown for a datasource connection in the Plans & Budgets workspace. 1. In the left pane, open a workspace menu and select **Datasources.** 2. In the main page select **Create Datasource +**.

3. Enter a Datasource Name, then for Connector Type select Postgres. 4. Under Configure connector, provide the following inputs: * **Host Name** - The hostname for the datasource (check the browser address bar; you want the string between **//** and **/**). Example: `https://`**`host.na.me`**`/` * **Database Name** - The name of the database you're connecting to. Each datasource can only connect to one database. If you need to connect to more than one, you'll need to create a datasource for each. * **Username** - Enter ***lightup***. * **Password** - The username account's password. 5. After entering the required settings and any optional settings that apply, below the **Configure connector** section select **Test Connection**. 6. After a successful connection test, select **Save**. 7. Your new datasource appears in the list of available datasources. By default, these are listed in alphabetical order, so you might have to scroll or change the sort order to see your new datasource. Here's our example new datasource connection, successfully tested:

## Add schemas to Explorer After you have a datasource, its schemas must be added to Explorer so you can begin data quality monitoring. 1. Select the datasource in the Explorer tree. 2. In the main page, select the **Manage Schemas** tab. 3. Turn on the **Active** toggles for each schema that has data you want to monitor.

Note that each schema is scanned when you activate it, which can take some time to complete. ## Set up data profiles To better understand your data and help enhance your data quality analysis, enable data profiling. Data profiling provides a static analysis of your tables that you can view in the Explorer tree. Data profiles are currently generated only for string, numeric, and time-oriented columns. Other data types such as boolean will be absent from the results. You enable data profiling when you add data assets to Explorer. The initial data profile uses up to one million rows to create each table's data profile, as follows: * For a table with under a million rows, the data profile covers the entire table. * If Lightup can determine which timestamp column to use for time-range queries and the table has at least a million rows, the data profile covers the latest 30 days worth of data. * If Lightup can't tell which timestamp column to use for time-range queries and the table has at least a million rows, the data profile covers one million randomly-selected rows. If you aren't sure what types of data quality checks you want, we recommend you enable data profiling to learn about your data. Reviewing these data profiles will give you the info you need to proceed with your data quality journey. 1. Select the datasource in Explorer. 2. On the **Summary** tab, select **Enable All Data Profiles**. This will create data profiles for all the tables in the schemas you added from the datasource, and may take some time.:

### Review data profiles After data profiles are generated, you can review them to help you plan your data quality analysis. For example, reviewing a table's profile will clarify which columns can be used for timestamps in metrics. When you select a table in the Explorer tree, its **Data Profile** tab displays the table's current data profile:

Each data profile is laid out in sections: Data profile, Table Schema, Timestamp Analysis, Sample Data, and profiled columns (section not labeled). Review the sections of the data profiles to determine which tables you want to start monitoring. For more details, see [Profile your data](doc:set-up-data-profiling). Identify a table to focus on first, then proceed. : * The Table Schema section shows the timestamp column, numeric value column and other columns making it suitable for any metric type.. * There are several other columns that are suitable for metric slices. Slices let you group by the values of a column in a metric to automatically create separate datapoints for each slice value. * You can also view the detailed profile for each column.

## Enable auto metrics After you've identified a table, consider enabling its auto metrics. Auto metrics provide a simple way to get started understanding your data quality. When you add data assets, you can enable their auto metrics. Lightup makes it possible for you to enable auto metrics at various levels in your data hierarchy, selectively or in bulk. There are two kinds of auto metrics: metadata metrics that query system data, and deep auto metrics that query your business data. For our example, we'll enable all the auto metrics on the table we've identified. There are several ways to do so— we'll use the Table Health tab. 1. With the table selected in Explorer, select its **Table Health** tab.

2. Auto metrics are listed at the top of the Table Level Metrics section. To enable them: * Select **Enable** beside the name of a metadata metric. * Select **Configure table** beside the name of a deep auto metric— this is required because deep auto metrics query the table's data. Review the table configuration and make sure the fields are correctly populated, then click **Confirm and save** to return to the Table Health tab. Then, select **Enable** beside the name of the auto metric. After you've enabled auto metrics, their names become menu links with three actions: View in Explorer, Monitor Metric, and Disable Metric. Note that at first the auto metric health indicators— the boxes just below the Auto Metrics section label— are gray. This is because until you add at least one monitor to a metric, its health is unknown.

## Create a metric It's a good practice to add monitors to your auto metrics and let them run for a while to establish basic monitoring of the timeliness, accuracy, and completeness of a table's data before you begin creating your own metrics for it. To keep things simple for this Quick Start, we'll go ahead and create a simple aggregation metric that measures the table we configured. 1. In the Explorer tree, select the data asset that you want to measure. 2. On the asset's Explorer menu select **+ Create Metric**:

3. A new metric opens at Step 1 of the metric configuration form. Under **Metric Info**, select **Aggregation**. 4. Enter a **Metric name**, then select **Next** at the top right. **Step 2 (Configure Metric)** appears in the main form. 5. Under **Aggregation function**, select **Sum**— you may have to scroll down or use the arrow keys to find it in the list. 6. Select **Next**. **Step 3 (Preview)** appears in the main form. Previewing your metric shows you what it would look like with the current configuration, but it's optional so we'll skip it. 7. Select **Next**. **Step 4 (Related metrics)** appears in the main form. You can add related metrics to make them available in your analysis of incidents from the metric you're creating. It's optional so we'll skip it for now. You can always edit the metric later and add related metrics then. 8. Select **Save**. The configuration form closes and your new metric opens in Explorer. ## Add monitors After you create a metric, you add monitors to detect when the metric value is out-of-bounds. The quickest way is to use the metric's Monitors menu. We'll use the following procedure to add a monitor to the metric we created. 1. With the metric open in Explorer, select its Monitors menu. 2. Choose **+ ADD**.

Lightup opens the **Add monitor** modal, which usually has two tabs: **Threshold** and **Anomaly Detection** (both tabs depicted below).\ ![](https://files.readme.io/5372bf4-auto-monitor.png)Note that some metrics do not support both tabs, because they don't support both underlying monitor types. For example, Compare metrics only support manual thresholds, so the Anomaly Detection tab does not appear in the Add Monitor modal for them. 3. Select the **Anomaly Detection** tab, then click **OK**. Lightup adds a new monitor to the metric using the [default settings for an aggregation metric](default-monitor-settings#aggregation-metric). ## Managing incidents Congratulations! You've just prepared a data asset for data quality managment and created a monitored metric in Lightup. Because you chose to use anomaly detection, your monitor will intelligently determine when the metric's values are out-of-bounds and generate an incident accordingly. Because you just set it up, you won't have any incidents yet. To learn more about handling data quality incidents, see [Manage incidents](manage-incidents) and [Analyze an incident](analyze-an-incident). # Learn more about Lightup Now that you've seen the basic end-to-end flow, you might want to learn more about your options: * [Explorer overview](doc:explorer-overview) * [Metrics overview](doc:metric-overview)