# Metrics overview

Metrics are how you measure the quality of your data. Some metrics require no configuration— you can use them out of the box to gain fundamental insights about your data health. For the team that wants more customized data insights, Lightup supports a full spectrum of powerful metrics, from low-code template-driven metrics that answer common questions, to custom SQL metrics that answer questions specific to your business requirements.

# Metric types

Lightup offers a full range of metrics, of four main types:

- Column metrics answer questions about specific columns, and have the widest variety of configuration options.
- Comparison metrics let you compare two tables (directly or through comparable metrics), helping you ensure ETL pipeline data quality.
- Table metrics answer basic questions about your table's health, including low-cost metadata checks.
- SQL metrics let you answer arbitrary questions about your data by writing SQL.

See Create and edit metrics for details on how to create each type.

## Column metrics

### Aggregation

Calculate an aggregate function on a column's values for a set of rows

Metric | Description | Column |
---|---|---|

Avg | Returns the sum of values divided by number of rows. | Numeric |

Count | Returns the count of values. | Any (not recommended for ID columns) |

Count Unique | Returns the count of distinct values. | Any (not recommended for ID columns) |

Max | Returns the largest value. | Numeric |

Median | Returns a number for which half of the values are smaller and half are larger. | Numeric |

Min | Returns the smallest value. | Numeric |

Percentile | Returns the value at or below which the specified fraction of values in its frequency distribution falls. | Numeric |

St.Dev | Returns the standard deviation. | Numeric |

Sum | Returns the sum of values. | Numeric |

### Conformity check

Calculate the percent of rows that conform to a condition

Metric | Description | Column |
---|---|---|

Contains/Does not contain string | Compares selected column's values with input string. | String |

Equal/Not equal | Compares selected column's values with input. | Any |

Greater than | Compares selected column's values with input number and matches if the column's value is greater. | Numeric |

Greater than or equal to | Compares selected column's values with input number and matches if the column's value is greater. | Numeric |

In/Not in | Compared selected column's values with an input set of values. | Any |

In column/Not in column | Compares selected column's values with a second column's values. | Any |

Is decreasing | Checks that column values are always decreasing. Returns 100% when current value is less than the previous value, and 0% when it is greater than or equal to the previous value. | Numeric |

Is increasing | Checks that column values are always increasing. Returns 100% when current value is greater than the previous value, and 0% when it is less than or equal to the previous value. | Numeric |

Is unique | Checks for unique values. | Any |

Length equal to | Compares selected column's values with input number and matches if the length is equal. | String |

Length greater than | Compares selected column's values with input number and matches if the length of the column value is greater. | String |

Length greater than or equal to | Compares selected column's values with input number and matches if the length of the column value is equal or greater. | String |

Length less than | Compares selected column's values with input number and matches if the length of the column value is smaller. | String |

Length less than or equal to | Compares selected column's values with input number and matches if the length of the column value is equal or smaller. | String |

Less than or equal to | Compares selected column's values with input number and matches if the column's value is greater. | Numeric |

Match pattern/Does not match pattern | Compares selected column's values with input pattern. | String |

Match regex/Does not match regex | Compares selected column's values with input regular expression. | Any |

Null/Not null | Checks selected column's values for nulls. | Any |

### Distribution metrics

Calculate the distribution of a column's values

Metric | Description |
---|---|

Distribution | Measures the distribution of distinct values. |

### Null Percent metrics

Metric | Description |
---|---|

Null Percent | Measures the percentage of null values. |

## Comparison metrics

Calculate how much two similar objects match

Metric | Description |
---|---|

Compare aggregate | Measures the degree to which two aggregate metrics match. |

Row by Row | Measures the percentage of rows in two tables where the key columns match, and the values of attribute columns in those matching rows. |

## Table metrics

Measure characteristics of a table or its behavior

Metric | Description | Supported datasources |
---|---|---|

Data delay | Measures the time elapsed since the most recent data arrived. | All |

Data volume | Measures the number of rows loaded. | All |

Column activity | Counts changes to columns (columns added, altered or dropped). | All |

Update Delay | Measures the time since the last data write. | BigQuery, Databricks, Incorta, MS-SQL, Redshift, Snowflake |

Byte Count | Counts the total number of bytes. | Databricks |

Row Count | Counts the total number of rows. | BigQuery, Incorta, MS-SQL, Oracle, Postgres, Redshift, Snowflake |

Note that Column activity, Update Delay, Byte Count and Row Count are metadata metrics.

## SQL metrics

SQL metrics support any valid SQL.

Updated 3 days ago