Cloud ETL

A magnifying glass over a bar chart with an upward arrow, accompanied by two gears in the background, symbolizing analysis and improvement. The image has a blue background.

CLOUD ETL GUIDE

Qlik Compose®: Cloud ETL as Part of DWH Automation Qlik Cloudbeam: Cloud ETL from Any Data Source

Introduction to Cloud ETL

Enterprises are increasingly drawn to cloud-based data warehousing services, with Amazon Redshift, Google BigQuery, Microsoft Azure, and Snowflake leading the growing market. Cloud data warehousing saves time and money compared to traditional on-premises data warehousing.

However, you will face challenges around cloud ETL. Cloud ETL entails extracting data from diverse source systems, transforming it to a common format, and loading the consolidated data into the data warehouse platform to best serve the needs of enterprise business intelligence, reporting and analytics. Fortunately, there are solutions which simplify and automate the cloud ETL process, bringing the benefits of cloud data warehousing within reach of any organization large or small.

Qlik Compose®: Cloud ETL as Part of DWH Automation

Available through the Amazon Web Services Marketplace, Qlik Compose is an innovative data warehouse automation solution that empowers data architects and data warehouse administrators to design and implement data warehouses and data marts without doing any manual coding. Bundled with Qlik Compose for Amazon Redshift, Qlik Cloudbeam makes it easy to configure and execute data migrations from nearly any type of source system into the staging area of a cloud data warehouse – again with no manual coding. Seamlessly integrated and remarkably easy to use, Qlik Compose and Qlik Cloudbeam accelerate and automate all aspects of cloud ETL for cloud data warehousing.

With Qlik Compose for Amazon Redshift you can create custom data warehouse designs through an intuitive design studio, or simply import a design such as Data Vault, Inmon, and Kimball. Qlik Compose then automatically generates the Amazon Redshift ETL code needed to transform data in your warehouse staging area to meet the requirements of your warehouse design pattern. With a click, you can create the warehouse structures and execute Redshift ETL tasks, monitoring the cloud ETL progress through a central console.

Cloud ETL is just as easy for data marts as it is for the data warehouse: Simply configure your fact and dimension tables by using the Qlik Compose star schema wizard, and Qlik Compose then automatically generates the cloud ETL command set that's needed to correctly populate the data mart.

Qlik Cloudbeam: Cloud ETL from Any Data Source

Qlik Cloudbeam -- the AWS marketplace name for Qlik Replicate® -- is a universal solution for moving data from nearly any type of source system into your cloud data warehouse staging area, where it can then undergo the transformations specified in the cloud ETL code that's automatically generated by Qlik Compose. With no manual coding, data warehouse architects and administrators can use Qlik Cloudbeam to configure and execute data feeds from cloud sources such as Amazon RDS or databases running in Amazon EC2; or by using an on-premises instance of Qlik Replicate to access nearly any type of on-premises source system such as relational databases, legacy data warehouses, or mainframes. Qlik Cloudbeam and Qlik Replicate support high-performance bulk loading and incremental change data capture (CDC) to power your real-time data warehouse and deliver timely analytics.

Along with fast and easy feeding of an Amazon Redshift data warehouse, Qlik Replicate also supports cloud data transfer to other destination systems such Azure and Google Cloud, and even data warehouse Hadoop platforms such as AWS Elastic MapReduce – all through a single unified data integration solution.