Explorer enables you to navigate your data asset hierarchy and see both data profiles and data quality metrics associated with those data assets.

This section details how to populate the Explorer tree by enabling your data assets via a set of slide-in "Manage " panes called the data asset management hierarchy (or the management hierarchy, for short).

Within the management hierarchy you can easily navigate by clicking the name of an object listed in the pane to go "down" the hierarchy, and by clicking Back to go "up". This can be easier than clicking on the data asset in Explorer and using its Actions menu. Either way works fine, and you can go back and forth based on what works best for you.

Add datasources

At the top of the hierarchy are your datasources. To begin populating the Explorer tree, add your datasources.

Add schemas

To add a schema to your Explorer tree, select a datasource and then select Actions > Manage Schemas.

The Manage schemas pane displays the datasource's schemas, as shown below.

  • Use the Active toggle to enable the schemas you want displayed in the Explorer tree.
  • If you want Lightup to track table changes in a schema, enable the Activity autometric.
  • Bulk operations are available in all Manage panes. To enable or disable a setting for multiple data assets at once, select the check box in the left column for each asset you want to change. Then change the setting in any selected row to update all the selected data assets.

Add tables

To add a table to your Explorer tree, select a schema and then select Manage Tablesfrom the Actions menu. The tables are displayed in the Manage schemas slide-in pane. An example is shown below.

  • Choose the tables you want to work with and toggle on the Profile toggle if you want to do data profiling. Your tables will now display in the Explorer tree.
  • Choose the tables you want to work with and toggle on the Configure toggle if you want to create data quality metrics. This requires you to configure your table with a variety of defaults such as the timestamp column, the data lag (evaluation delay), and partition formats. View Table configuration for help on how to configure your table. These defaults are inherited by you data quality metrics. If you know this information and you are preparing to create data quality metrics, then configure your table. If you don't know this information and want to only do data profiling, leave Configure toggled off.
  • If you choose to Configure your table, optionally toggle on the three table autometrics: Activity, Data Delay, and Data Volume. For more info on these autometrics, see Autometrics

Add columns

To add columns to your Explorer tree, select a table and then select Manage Columnsfrom the Actions menu. The columns are displayed in the Manage columns slide-in pane. An example is shown below.

  • Make any columns you are interested in Active
  • If you have turned on data profiling for the table, you'll be able to see the column's data profile by clicking on the column
  • Change a column's type by clicking on the A or # and select the alternate type.
  • If you have configured the table for metrics
    • you'll be able to create metrics on the column
    • Optionally toggle on the three table autometrics: Activity, Distribution, and Null Percent. For more info on these autometrics, see Autometrics

Add autometrics

Lightup supports a variety of data quality autometrics. For details on these autometrics, see Autometrics. As mentioned in previous sections, these can be turned on and off via toggles at each level of the management hierarchy.

Most autometrics are at the table and column level, and require you to first configure the table. For details, view Configure your table for data quality analysis. The one exception is the schema level Activity metric, which tells you about changes in the tables in your schema. This can be turned on as soon as the schema is made active.