All About Acceldata Data Observability Cloud v2.7.0
June 7, 2023
10 Min Read
Acceldata Data Observability Cloud v2.7.0 Introduces Significant Databricks and Snowflake Observability Enhancements and Supports New Data Sources
Recently, we released the latest version of Acceldata Data Observability Cloud (ADOC). Version 2.7.0 is now generally available, and it provides enhancements to compute, data reliability, monitoring, alerting, and Databricks and Snowflake navigation and display capabilities.
The following video offers a snapshot of the enhancements, and more details on new capabilities and features are explained below:
Data Source and Data Plane
Check out Acceldata’s Hive integration and new data sources:
Enhanced Hive Integration: ADOC has been upgraded to seamlessly integrate with Hive, specifically the Enterprise version. Detailed information on this integration can be found in the Hive documentation.
Expanded Data Sources: ADOC now offers integration with additional data sources, including Teradata, Azure MSSQL, and Oracle. To learn more about utilizing these integrations, refer to the documentation for Teradata, Azure MSSQL, and Oracle.
In the 2.7.0 release, you’ll see a variety of major enhancements that will provide better experiences for Databricks Query Studio, Snowflake Query Studio, cost center and cloud cost control features, warehouse sizing recommendations, and others.
Databricks Query Studio: ADOC now offers a Databricks Query Studio experience, akin to Snowflake Query Studio. This exciting addition enables you to validate and monitor SQL queries executed by Spark apps within a notebook. For detailed insights, please refer to Acceldata’s Databricks Query Studio documentation.
Enhanced Databricks Guardrails: In the Databricks Compute tab, a new Guardrail page has been introduced, presenting visualizations for clusters that have been terminated due to inactivity. To explore this feature further, consult Acceldata’s Databricks Jobs Visualizations documentation.
Advanced Cost Center Management with Tag Pairs: ADOC now offers the ability to create chargeback-based cost centers using Tag Pairs. This feature enables you to accurately track charges for the entire account or specific subsets of resources by utilizing multiple key values. Check out our documentation about Cost Centers and Organization Units.
Streamlined Databricks Navigation: ADOC's Databricks navigation now facilitates efficient correlation of data across jobs and clusters. For details, consult the Databricks Visualizations documentation.
Enhanced Snowflake Query Studio: The Snowflake Query Studio now displays metrics like #Avg Schedule Frequency, warehouse names on hover, easier navigation to the Query Details page, and additional information in the heatmap chart, among other enhancements.
Projected Spend Budget Alerts: ADOC introduces the capability to receive alerts when your projected spend, calculated from current utilization, surpasses a predefined budget threshold. Stay informed about your forecasted expenses by exploring the Budgeting documentation.
Snowflake Warehouse Size Selection: You’ll now have the option to choose a Snowflake warehouse size based on intelligent recommendations which streamlines your decision-making process.
We’ve made significant enhancements to the data reliability capabilities of the Acceldata platform. These include better data cadence support, importing/exporting policies, service monitoring, policy execution, and others.
Customizable UDF Metadata for Failed Rows: ADOC now enables you to customize the feedback logic to UDT via metadata for various checks, so you can receive detailed feedback for failed records pertaining to Data Quality Policies. Check out User Defined Templates documentation.
Expanded Data Cadence Support: ADOC now offers data cadence functionality for Files and has extended its support to include MySQL, PostgreSQL, BigQuery, S3, ADLS, Azure Data Lake, GCS, and HDFS as data sources. Get insights into configuring data cadence by referring to the Data Cadence Tab documentation.
Composite Policy Application: You’ll now be able to apply individual rules to specific subsets of data within an asset through the Composite Policy feature.
Enhanced Service Monitoring in Dataplanes: ADOC now provides real-time visibility into the ongoing logs of the analysis service executed on the dataplane side. Additionally, you can view the running logs of crawlers on data sources. See our Managing Data Plane documentation for details.
Monitors and Alerts
This version has enhanced monitoring functionality, and new recommendation and alerting capabilities.
Databricks Workspace Cost Monitor: In this release, we’ve provided Databricks Workspace Cost monitor, which tracks the cost at the workspace level for the current month. Our Stock Monitors documentation has details.
Enhanced Recommendations: ADOC now allows you to enable or disable recommendations directly from the Monitors interface, providing greater control over the recommendations displayed. Additionally, recommendations will automatically expire when they are no longer relevant. Here’s our Monitors List documentation.
Alerts for Assignee Action: ADOC now includes a new email notification template that ensures users assigned to an alert receive timely email notifications. Check out our Alerts documentation.