Review the latest updates for key data and analytics technologies and platforms including Snowflake, Databricks, dbt, BigQuery, Looker, Qlik, Tableau, and Power BI.

We know the product release notes from the vendors can be very detailed and overwhelming. So, we have outlined the major product updates you need to know about for Q1 2022, how these updates can impact you, how they can be applied, and other major news for key technologies within the data and analytics space.

Snowflake

Product Release Updates

  • Release 6.0 (1/19): Snowflake Scripting – Preview: “Snowflake Scripting is an extension to Snowflake SQL that adds support for procedural logic. You can use Snowflake Scripting to write stored procedures in SQL.”
    • SQL Scripting is a step forward in usability and capability. It allows SQL logic to be executed directly, no longer requiring use of stored procedures wrapped in JavaScript. This increases the number of projects that can be performed entirely in Snowflake without third-party tools.
  • Release 6.1 (1/24): Unstructured Data Support – General Availability: “Enables users to access, load, govern, and share unstructured files of data types for which Snowflake has no native support, including some industry-specific types. Support for unstructured files adds to the existing robust support for structured and semi-structured data.”
    • This update reduces the need for additional tools alongside Snowflake to manage and utilize unstructured data.
  • Release 6.5 (2/8): External Table Support for Delta Lake — Preview: “With this release, we are pleased to announce preview support for Delta Lake in external tables. Delta Lake is a table format on your data lake that supports ACID (atomicity, consistency, isolation, durability) transactions among other features. All data in Delta Lake is stored in Apache Parquet format. Query the Parquet files in a Delta Lake by creating external tables that reference your cloud storage locations enhanced with Delta Lake.”
    • This improves Snowflake’s flexibility to be deployed in architectures alongside Databricks.

Other News 

  • Snowflake released new industry-specific data clouds. These are solutions to accelerate capabilities of organizations in financial services; healthcare and life sciences; retail and CPG; advertising, media, and entertainment; public sector; education; and technology.

Databricks

Product Release Updates

  • Version 3.6 – Syntax highlighting and autocomplete for SQL commands in Python cells: Syntax highlighting and SQL autocomplete are available when you use SQL inside a Python command, such as in a spark.sql command.
    • These features help to reduce developer errors and increase speed of development of teams working in Databricks.
  • Public Preview – Unity Catalog: The Unity Catalog is a cross-workspace metastore which will help improve centralized data governance for the Lakehouse. One of the main features of the Unity Catalog is the Delta Lineage view, which will auto-capture and map the flow of data into the Lakehouse (across all languages down to the table and column level).
    • This will bring a level of governance and oversight to Databricks that has previously been missing.
  • Public Preview – Delta Sharing: Delta Sharing is a feature of the new Unity Catalog and will allow for Databricks-to-Databricks verified data sharing. It will allow incremental data changes to be shared using Delta Change Data Feed and has a simple UI to create and modify shares and recipients.
    • This feature adds capabilities of data sharing features similar to existing solutions offered by competitors like Snowflake.
  • Public Preview (AWS only) – Serverless DBSQL: Serverless DBSQL sets aside managed server instances which are always running, patched, and upgraded automatically to allow customers to access instant compute Power.
    • This reduces wait times for clusters to spin up. Databricks is also working to improve first query performance up to 3x using serverless DBSQL.
  • Public Preview – Databricks SQL: Databricks SQL is undergoing new updates in public preview, including a tabbed editor, advanced autocomplete and syntax error highlighting, allowing for multiple concurrent executions per tab, and sharing past executions to other users.
    • The same queries can be run from notebooks using SQL, however the Databricks SQL view is a more traditional SQL query editor which will allow Databricks to compete more fully with Snowflake and other cloud DWH competitors as it provides a simpler experience for querying directly against the Lakehouse.

Other News 

  • You can now create Databricks workspaces in your VPC hosted in the AWS ap-northeast-2 region. You can use a customer-managed VPC to exercise more control over your network configurations to comply with specific cloud security and governance standards your organization may require.

dbt

Product Release Updates

  • Version 1.0 (stable release): dbt version 1.0 offers ease of upgrades and is backward compatible. It serves as the foundation for new functionality to be built upon in future releases. This marks a significant milestone in product maturity as ideas that previously seemed ambitious—including artifacts, advanced testing, Slim CI, and more—have materialized as incredible features in dbt v0.21 and earlier releases.
    • As product manager Jeremy Cohen mentioned at Coalesce 2021, this means the dbt core features have reached a point where they are fast, stable, intuitive, extensible, and maintainable. After upgrading to dbt 1.0, you can be certain that your future upgrades will be simple and intuitive from that point on. A few future ideas on the roadmap for these upgrades are column level lineage, support for sharded tables, built-in linting, column level Slim CI, and eventually dbt-SQL.

Other News 

BigQuery

Product Release Updates

  • General Availability – Qualify Clause: The qualify clause is key to make code more concise. This has eliminated redundancies in code while executing window functions for transformations.
    • One of our clients (Driven Brands) will use the Qualify clause to automate deduplication of critical source data in a view within BigQuery.
  • Preview – Table Clones: Cloning tables allows a cost effective, light-weight way to store copies of tables and test changes to a table. You are only billed for storing the cloned table once it differs from the base table.

Other News 

  • A BigQuery Reliability Guide was published in Q1 2022 to help customers create solutions that correctly contemplate reliability requirements for specific use cases. This helps customers determine the best service within GCP for their solution rather than trying to shoehorn BigQuery for everything when it is not the optimal service. They included areas of planning for import reliability, query reliability, read reliability, and disaster planning.

Looker

Product Release Updates

  • API 4.0 is generally available: Introduces new functionality that was previously inaccessible via the Looker API, like copying dashboards.
    • Copying dashboards previously could not be done via the API without some heavy coding. This turns what used to be a 50-60 line script into about 2-3 lines of code. We immediately began using this for one of our customers, but could see it being implemented any time a regular set of dashboards needs to be copied into a new folder.
    • The ability to use Looker’s API to automate administrative and embedding use cases sets it apart from its competitors. Tedious management of user permissions and authentication can be automated down to a few keystrokes or scripts.
    • From an administration standpoint, Looker is an InfoSec dream. Permissions can be set up via a script and repeated multiple times, and pipelines for copying dashboards/checking user access can be established. The sky’s the limit with the API, but it blows the competition out of the water when it comes to administration.
  • LookML Dashboards can be moved to any content folder: Allows for more logical storage of LookML dashboards, especially for Looker instances where multiple companies are logging into the same central instance.
    • The LookML Dashboards update is a step toward fully version-controlled dashboard creation, where changes to a dashboard can be tracked over time, so users can always be confident that the dashboard they’re seeing is the correct, working version.
    • Before this update, LookML Dashboards all had to stay in one folder, which was not manageable if working on an instance with hundreds of LookML dashboards. The concept of connecting a Looker dashboard to a Git service is something that sets Looker apart from the competition, and benefits users and developers alike, knowing that changes can be tracked and rolled back, if needed.

Other News 

  • Looker is sunsetting its certification program to align more with Google’s ‘badge’ system.

Qlik

Product Release Updates

In Q1 2022 the Qlik SaaS product has continued to add additional functionality and integration with their recent acquisitions.

  • Qlik Forts: Allows organizations to keep their data and application on premise, while taking advantage of SaaS functionality and upgrades.
    • Qlik Forts will provide an option for those clients that are hesitant to keep all of their data and applications on a cloud environment. This opens the tool to some clients that have security or regulatory concerns.
  • Qlik Automations: Qlik automations has become a more robust feature in the SaaS environment. Templates have been created to provide a library of ways that automation can be utilized to take analytics outside the SaaS environment and greater control within the SaaS environment.
    • Qlik Automations/templates is going to provide customers with a wide range of ways to create API code that will allow the organization to take analytics outside of simple dashboards.

Tableau

Product Release Updates

As of March 21, 2022 Tableau 2021.4 is the latest version of Tableau.

  • Virtual Connections: A content type in Tableau Server/Online that lets you create and manage access to data sources within Tableau Online. Unlike a standard published data source, virtual connections allow you to securely embed service account credentials and define data policies for row-level security, all within the Tableau Online platform. This greatly simplifies data management and access, and makes it much easier to implement and manage row-level security; previously you needed to create custom logic within a Tableau workbook, publish an entirely separate workbook data source, and manage control offline.
    • Virtual Connections is an important new feature because it automates and simplifies data management and security processes. This feature bridges the gap between the ‘core’ Tableau offering (Tableau Desktop, where you make your dashboards) and Tableau Online/Server, where you share reports. Having this single connection ensures all users are accessing fresh data sourced centrally from managed extract refresh schedules. Administrators save time by making database changes only once in the virtual connection rather than in every connection per Tableau content (data source, workbook, flow).
  • Copy and Paste: Added functionality to Tableau Desktop that allows you to copy and paste workbook elements from one page to another, even across workbooks.
    • This feature accelerates development of reports and dashboards, and it encourages and simplifies design consistency.

Other News 

  • Tableau for Slack: allows users to share a vizualization generated in Ask Data directly to Slack, putting data at the center of every conversation. Streamline communication and collaboration on one platform to move business forward.

PowerBI

Product Release Updates

  • In the February 2022 Power BI update, Microsoft released the ability to create, modify, and execute Power BI Deployment Pipelines directly from Azure DevOps using the new Power BI automation tools extension. This open-source Azure DevOps extensions removes the need to use API or scripts to manage these pipelines.
    • The new Power BI automation tools extension for Azure DevOps allows for organizations to quickly incorporate Power BI dataset and report management into their existing Azure DevOps pipelines. Enabling continuous integration and continuous delivery (CI/CD) for Power BI. Automating this management process for Power BI datasets and reports provides decisions makers the latest information more quickly and accurately—removing error introduced by manual processes.
    • With the simplicity of incorporating this capability, Power BI developers can automate delivery of these different reports and datasets to users, unlike other BI tools. Managing workspaces, user access and promotion processes of these Power BI objects further enhances the amazing list of features and capabilities of Power BI.
    • Incorporating the Power BI automation tools extension into existing Azure DevOps pipelines is fast and easy. There is no longer a need for someone to manual promote datasets and reports, instead freeing them up for other tasks. This also significantly reduces potential error caused by manual promotions.

Other News 


These updates are current as of Q1 2022. Keep an eye out for quarterly updates on technologies within the data and analytics space.

Patrick Vinton Patrick oversees R&D and is responsible for the technical direction of Analytics8. When he's not working, he's probably playing with his 2 sons. If the kids are with the babysitter, he's sharing a bottle of wine with his wife while binging on Netflix - probably a documentary or historical drama.
Tony Dahlager Tony is Analytics8’s Managing Director of Data Management, leading sales, marketing, partnerships, and consulting enablement for our data management service line.
Kevin Lobo Kevin is our Managing Director of Analytics and is based out of our Chicago office. He leads our Analytics service line, overseeing its strategic direction and messaging, while ensuring delivery of high impact analytics solutions for our clients. Outside of work, Kevin enjoys spending time with his wife and daughter, as well as running and live music.
Subscribe to

The Insider

Sign up to receive our monthly newsletter, and get the latest insights, tips, advice.

Thank You!