A Monumental Year for Qlik’s Data Integration and Quality Business

At Qlik World 2023, Mike Capone stood on stage and proudly announced the acquisition of Talend to form Qlik’s Data Business Unit. The Data BU had the charter to combine the best of Qlik Data Integration with Talend and Stitch Data to deliver market-leading data integration and quality solutions.

The data integration space has seen many twists and turns over the past year, with huge acquisitions and rumors of buyouts, but nothing had more of a ground-shattering impact than the debut of ChatGPT and other Large Language models. Gen AI has gone mainstream, and virtually every company I speak to has a strategic mandate to use AI for competitive advantage.

Moreover, I tell every customer, “Your AI success depends on having accurate, timely, and high-quality data in a readily consumable format.” and that philosophy was our North Star for developing everything I showed at this year’s Qlik Connect conference. We have a long list of product achievements but let me start with the launch of Qlik Talend Cloud.

Introducing the Qlik Talend Cloud Vision

Qlik Talend Cloud, the first combined offering with Talend, is a pivotal step in our journey. It not only sets the stage for future SaaS data integration, quality, and AI innovations but also embodies our commitment to delivering cutting-edge solutions. Based on the Qlik Cloud Infrastructure with over ten thousand tenants, a multi-region presence and over a dozen certifications, this platform serves as a beacon for our new innovations. In addition, we will continue to migrate more of the Talend Cloud capabilities over time too.

With the introduction of Qlik Talend Cloud, we are ushering in a wave of innovation to the market. This new platform empowers you to swiftly build and deploy AI-augmented no-code ELT pipelines that distribute trusted data assets throughout your organization. Now, let’s delve into the features and key themes of Qlik Talend Cloud that bring tangible benefits to your data integration and quality processes:

• Innovative Data Products — Data products are highly curated, trusted, and reusable data assets that safely bridge the gap between data producers and consumers. These collections of datasets, business-approved metadata, and domain logic form the backbone of powerful data apps like recommendation engines, predictive machine learning, business analytics, and generative AI. For more details about this compelling approach to producing curated data for various domains with greater speed, improved quality, and auditable data governance.

• Curated Data Marketplace — Another component of our data product vision is to deliver a curated data marketplace that drives the collaboration and consumption of data products. We envision infusing natural language search in the marketplace to make discovering, understanding, and sharing data products extremely intuitive and easy. Follow this link for more information about Data Products and the Marketplace.

• Modern Data Engineering — Our unique no-code/pro-code approach to defining data pipelines and transformation logic also took a quantum leap forward. Thanks to some embedded AI technology, the connector factory got a significant upgrade, and we now support over 170 SaaS data sources, with more rolling out monthly. Incidentally, if you don’t see a data source connector you need, just request it here.
  
  We’ve also folded SaaS data-loading technology from Stitch Data into Qlik Talend Cloud. The new starter package is perfect for customers just wanting to load SaaS data into their data warehouse. The great thing about the starter package is that it also provides a seamless path to more advanced features like SAP connectivity, data transformation, and data products. Watch out for the launch of Qlik Talend Cloud Starter Package and other editions later this year.
  
  While I’m talking about data transformation, the introduction of Transformation Flows massively upgraded those capabilities, too. The simple drag-and-drop interface based on the Talend Cloud pipeline designer makes creating visual transformation flows easy for both data engineers and non-SQL experts alike. You can use transformation flows as an alternative to custom SQL code as part of your ELT data pipelines.
  
  

  

  
  
  Qlik Talend Cloud converts transformation flows to SQL statements, orchestrates them, and pushes them to your data platform of choice for runtime execution. Snowflake, Databricks, Google Big Query, Azure Synapse Analytics, and Microsoft Fabric are all supported.
  
  We now have a full range of data transformation features to suit every data engineer, regardless of skill. You can create fully automated data marts with point-and-click features, and transformation flows using drag-and-drop capabilities. If you don’t have the skills to code your own SQL, you can also use our new AI Assistant, which generates SQL transformations from natural language prompts. Follow this link to learn more about data transformation.

• AI Innovation – We’ve added many productivity and infrastructure features to help you roll out various AI initiatives. You can use custom SQL to collate data sets for training machine learning (ML) models. Later in the year, we will enhance that functionality by adding a transformation flow processor that specifically supports Databricks AI functions. We’ve also added support for vector databases and multiple LLM’s that help you build data pipelines to support Retrieval Augmented Generation (RAG) applications. Once again, we’re innovating by providing the capability to use structured and unstructured data sources for your enterprise corpus. Concurrently, we’re also expanding our support for Snowflake Cortex. Finally, we’ve also added a chat interface to allow Data Engineers to “talk” to the data pipeline as they are building. This helpful co-pilot-like feature accelerates development tasks such data quality rule building, column standardization and data exploration. For more information about the new AI features, check out this link https://www.qlik.com/us/products/qlik-talend-cloud.
  
  

Pioneering Trust Score for AI

I’m also proud to announce another AI innovation called the “Trust Score for AI”. This unique product and services offering will help you establish and monitor the key quality metrics for the data used in your AI projects. The Qlik Trust Score for AI is a system designed to assess the trustworthiness of an AI dataset by providing a single, easy-to-understand score that reflects the overall quality of your AI data.

Follow https://ai.trustscore.talend.com to experience the trust score with a sample dataset or your own data!

Enhancements to Our Client-managed Portfolio

Up to now I’ve talked about Qlik Talend Cloud, but I want to emphasize that we continue to invest in our client-managed products. We understand the reality of enterprise deployments and recognize that not all your infrastructure or workloads run in the cloud. We aren’t forcing you to migrate to cloud, like many other vendors in the industry, and we continue to support your choice of where you decide to run data processing workloads. I can’t list all our product updates from the past year, but here are a few highlights:

• Talend Studio Enhancements – We’ve given the Talend Studio UI a fresh modern look and feel that will definitely appeal to all developers.

• Gen AI pipelines in Studio – We’ve also added new components to support vector databases and LLM embeddings for RAG-based requirements, continuing the theme of weaving AI capabilities throughout the platform.

• Snowpipe Streaming – Snowflake Cortex isn’t the only innovation we’ve announced over the past year. Qlik Replicate also released support for Snowpipe Streaming which is a new method for loading data into Snowflake, directly into a Snowflake table as opposed to Bulk loading, helping to reduce latency and lowering costs.

• FIPS Compliance – As of Qlik Replicate May 2024 release FIPS compliance is supported as part of the standard setup routine.

• Enhanced API Support – Finally, we’ve extended Qlik Enterprise Manager APIs to now support the ability to add and change tables within specific tasks.

Conclusion

It’s been an exciting year leading the Data BU and I couldn’t be prouder of our vision, innovation, and products we’ve released this week. As we look towards the future, it's clear that the strides we've made in ensuring data integration and quality solutions help foster cutting-edge AI developments, we can be confident that these powerful tools will continue revolutionizing how we work and live. We’re on the right path and have been validated by customers and industry analysts alike so stay tuned for even more exciting advancements in the coming year!