4.0 (4/17/24)

Features

• Low code metrics now support more complex where conditions - you can specify individual numbered conditions and then choose to AND, OR, Custom combine them together. Custom will let you specify a complex combination, such as (1 AND 2) OR 3
• Casting support was enhanced across all datasources (except file-based sources) to allow automatic casting of numeric to string data types and vice versa. This will affect results in the data profile, distribution metrics, and available low-code operators.

Improvements

• Custom schedules are now available on full-table metrics
• A navigation breadcrumb trail was added to the incident page
• Error messages (such as when debugging a custom SQL validation) now have a copy icon to copy the error text to your clipboard

3.25 (3/31/24)

Features

• SAP/HANA datasource support is available - please contact support if you would like this enabled in your environment.
• Column masking is now available! From a table's 'Manage Columns' interface, you can choose to mask columns, preventing them from outputting raw data within Lightup (in data profiles, sample data, and failing record queries).

Improvements

• Introduced two new toggles to the Query Governance section:
  • Enable Data Storage: Allows Lightup to save row level details currently used in storing row-by-row comparison failing records. This data can be viewed from the metric and downloaded as a csv.
  • Enable Data Samples: Disables capabilities that display row level data (sample data and failing record data)
• Increase max slice count to 60k
• When bulk editing metrics/monitors/incidents a selection banner now shows the number selected across pages and a quick clear option.

Changes

• Missing value filling only supports null and zero filling. Previously we supported nan which was found to be no different than null in practice. All metrics that had nan configured will be migrated to using null missing value filling.
• Minor efficiency improvements and security patches.

3.24 (3/14/24)

Enhancements

• Multi-column slicing: you can now leverage multiple columns when adding slicing to a metric.
• List search improvements: The list search has been overhauled on metric, monitor, and incident list pages to support more fields. You can add filters from the search dialog box by typing and/or scrolling

Bug Fixes

• Incident validations where no data is present are no longer marked as successful
• Added infrastructure monitoring to identify stuck alert channels

3.23 (2/29/24)

Enhancements

• The left pane is now resizable via drag.
• An optimization has been put in place to avoid “waking” the datasource when a SQL metric is edited. Previously, editing a SQL metric would access the datasource to identify the values for the Columns input. This access will no longer be made when you first edit a SQL metric. Instead, if you need to view the columns, at the time you open the Columns dropdown you’ll be prompted to first validate your SQL.
• Query scope for metadata metrics is now displayed as “metadata”, to indicate that the scope of the data queried is from metadata tables.
• System event notifications now include the workspace name.
• Minor updates to format of the incident Activity tab:
  • Comments are now prefaced with “Comment:”.
  • Activities now include the name of the user who generated them. (System-generated activities contain no username.)

3.22 (2/15/24)

Enhancements

• If a datasource is paused by a schedule, Connection Status in the Datasources List will now reflect this status:

• The incident activity tab of incident analysis now contains the full history of an incident. This includes when it started, ended, any status changes or validation performed, and comments.

• The BigQuery datasource now supports views.
• Activity metrics can now be deleted, similar to all other metrics. Note: improvements in this area are still underway. In a future release you will be able to edit, pause, and resume these metrics.

Bugs

• An issue was fixed that restricted the user from specifying a limit on a failing record query on a full table metric. This has been fixed.

3.21 (2/1/24)

Enhancements

• Query history: You can now access the query history for a metric from both Explorer and Incident Analysis. From Explorer, click on the Metric menu to find the Query History menu item.

Similarly from incident analysis, click on the incident’s “...” menu:

• Filtering for slices with incidents: On a sliced metric, you can now select “Only Show Incidents” in Explorer to filter for only slices with incidents:

• Failing records
  • Failing records generated from a failing records query can now be sorted by column
  • You can now download up to 10,000 failing records. To specify the number of records to download, enter a number in the new Limit input box, click Run Query, then Download. Note that the records beyond 100 can be downloaded, but cannot be viewed in the UX.

• Table Activity, Column Activity, and Category Activity metrics:
  Previously these metrics did not show up in the metric list - you will now see them there. You can additionally view details of the metrics via “Edit metric”. Note: improvements in this area are still underway. In a future release you will be able to delete these metrics from the metric list and edit the metric from “Edit metric”. Currently you can edit some fields but you will not be able to save the changes.

• Opsgenie: You can now integrate Lightup with Opsgenie

• Databricks Unity: Databricks Unity catalog support has been added to Databricks Partner Connect

3.20 (1/18/24)

Enhancements

• Assignee field for Jira integration can now be selected via an autocomplete field that searches for assignee by name.
• Up to 10,000 failing records can now be downloaded to a csv file.

Deprecations

• We are no longer supporting Enterprise on-prem Jira servers (which has been end-of-lifed). Only cloud Jira is supported.

3.19 (12/29/23)

Enhancements

• Lightup’s new System Events panel provides an improved user experience for viewing asynchronous events that occur during system operation. Events are categorized by level - Error, Warning, or Info. You’ll find the System Events menu under your workspace in the left panel - where “Alerts” used to be. To see details of a system event, click on details and a slide in will appear with details of the errors. Fields in the “Info” section of the details slide in can be clicked on to take you to the object referenced.

You can be notified of system events in a manner similar to the notification system for incidents. From the System Events Configuration tab, select Error, Warning or Info and then select an alerting channel.

The Lightup System Events documentation contains more details, including the list of events that are logged in the System Events panel.

• The Schedules UX has been improved - it now provides a more intuitive UX, particularly for the case where the schedule “never ends”
• Last login has been added to the admin Users page, allowing you to see when a user last logged in to Lightup.
• Clicking on Edit Datasource in Explorer now takes you directly to editing the datasource. Previously it took you to the datasources list page.
• Issues related to paused datasource causing an error popup have been fixed. If a datasource pause causes a query to fail (for example, sample data during preview or in a data profile), the chart will now reflect the reason, rather than an error popup.
• The UX for filling in the lower and upper thresholds in the manual threshold modal has been improved.
• Query Governance “Enforce Sort Key Timestamps” language will now reflect the terminology of the datasource.
• A new design for toasts has been introduced. Rather than the red outline box that disappears, you will now see a “color block” popup which requires you to click on it to close.

Bugs

• If incident validation fails after three attempts, an error will be displayed and the metric will be placed into an error state. Previously Lightup continued to try to run validation.