Lightup is a next-generation data quality and data observability platform. It empowers your team to quickly and easily perform continuous, comprehensive data quality checks. Workflows that are currently manual become automated, allowing you to achieve complete data coverage across all of your data assets, across all cloud and hybrid environments.

Data profiles for quick insights

Data profiling is fundamental to data quality management. Knowing the shape of your data is crucial when you want to measure its quality. Lightup data profiling simplifies data quality analysis by illuminating that shape.

A data profile is a static analysis of a table. If the table is small, you can profile the whole table. For larger tables, you can profile a specified time range or a randomly selected set of rows.

Data profiling requires no input from you. You simply turn it on, and Lightup generates data profiles to help you understand your data. See Profile your data for details.

Data quality monitoring for ongoing quality management

When working with data, a variety of questions come up about the completeness, timeliness, and accuracy of your data. For example:

• Were my tables updated on time?
• If they were updated on time, did the expected volume of data show up in the tables?
• Was the data that appeared valid?
• Were there too many null values?
• For a categorical column, are the categories those that are expected, with the distribution expected?

Lightup enables you to answer these data quality questions via an automation platform that provides "always-on" data quality monitoring. It runs in the background, and generates routable alerts when incidents occur. See the data quality analysis Overview for details.

Slices to streamline DQ for your business structure

Lightup supports slices to help you streamline data quality analysis to match the organization of your business. Slices let you define a metric so that each value of a selected column (the slice column) is measured separately, using the exact same query except for the slice value. Then you can monitor each slice separately, defining different expectations and detecting any incidents on a per-slice basis. For example, you might think about your sales by brand, market, or channel. Any columns that identify these groupings would be a good choice to use for slices.

Lightup enables you to understand data quality by slice, automatically detecting slices and tracking individual metrics and incidents for each. Slicing is a natural and powerful way to enable automated and granular data quality monitoring that represents the organization of your business data. See Slices for details.

Auto metrics for common questions

Lightup includes auto metrics to handle many common data quality checks. These are turn-key metrics that can help you start managing data quality quickly, forming a basis on which you can build custom, more complex metrics. See Auto metrics for details.

Create metrics to suit your needs

Answer more complex questions about your data's completeness, timeliness, accuracy, and other data quality dimensions, by creating your own metrics. From no-code to low-code to full-on SQL, Lightup's metrics give you power and flexibility to build and improve your data quality analysis. By adding slices to a metric, you can create one metric that measures the table's data quality and then independently track data quality for each value of the slice column. See Create and edit metrics for details.

Monitor by thresholds or with anomaly detection

Lightup separates the metric (which measures quality) and the monitor (which establish rules to check metric values). This lets you create as many monitors as you want for a metric— for example, if your metric is sliced, you can add a different monitor to each of them.

When creating a monitor, you can set threshold values manually, or use automated anomaly detection to easily monitor when your data changes from its historical norm. See Monitor a metric for details.

Powerful, flexible data quality management

With a wide variety of turn-key features as well as extensive customizability, Lightup empowers enterprises to innovate and make business decisions with confidence by providing data teams with the fastest and easiest way to create trust in their cloud and hybrid data at any scale.

About this guide

This Lightup User's Guide is here to help you monitor your cloud data. Try the Quick Start to jump right in or dig into more in-depth content in the pages that follow.