Query scope
Query scope refers to the amount of the source asset's data that is queried for one metric data collection. For metadata metrics, none of the asset's data is queried— instead, system metadata is queried, so query scope is Metadata and cannot be changed. For deep metrics, the Query Scope setting controls whether metrics collect data from a specific time range (Incremental) or from all rows of data (Full Table). Query scope affects which other Data Collection settings are available, so you typically set it first when you configure a deep metric.
Full Table query scope is designed to support dimension tables (those with no time-based fields), while Incremental query scope is designed to support tables where you want to progressively query consecutive time periods.
- All deep metrics other than SQL metrics inherit the same Query scope as their source table.
- Full Table metric queries may be compute-intensive/expensive on large tables.
- Charts of Full Table metrics are left-aligned: because there are no timestamps, no matter when data collection actually happens, it appears on the chart as occurring at the time it was scheduled to run (based on the polling interval and the polling delay).
- Metadata metrics' source data is updated during schema scans, both scheduled and manual.
The table below gives a summary of the different query scopes.
Query Scope | What's collected | Dependent settings | Recommended use case |
---|---|---|---|
Incremental | By default, only those rows with a timestamp inside the most recent aggregation interval are collected. If Data Collection Schedule is set to Custom schedule or Triggered, rows from all previous aggregation intervals that haven't already been queried by the metric are collected. | Agg. Interval, Agg. Time Zone, Eval. Delay, Timestamp, Time Zone | Fact tables (tables with a timestamp column) |
Full Table | All rows, every time the metric query runs. | Polling Interval, Polling Time Zone, Polling Delay | Dimension tables (tables with no timestamp column) |
Metadata | All relevant system metadata, every time the metric query runs. | None configurable | Set automatically |
Updated 10 months ago