When Elizabeth Barrett Browning wrote her love sonnets, she may not have been thinking about the largest ERP system in the world. Here at Qlik, SAP is just one of the many data sources we love, and we think about them every day.
This blog is our sonnet to all the SAP endpoints we have made available for you when it comes to getting data out of SAP.
Many SAP customers right now are full of fear, uncertainty and doubt with SAP moving the goalpost for accessing their data inside SAP systems. They are thinking “What can we do?, What can't we do?, It seems that we can't do anything”. Fear not, Qlik can set these fears at ease.
Whether you have an SAP runtime license or an SAP enterprise license of HANA, if you're still on Oracle, or if you're on HANA, rest assured we've got something that will work for you.
We have database level connections and application-based connections, in the graphic for this blog you can see that they're colour separated top and bottom. We will take a deeper look under the hood of these endpoints and I will also explain certain factors that you may want to consider based on your risk assessment on any SAP published notes that may exist around these endpoints.
Let’s start with our database level connections, all of these endpoints feature our best-in-class change data capture (CDC) technology.
SAP HANA database, with trigger-based change data capture.
SAP HANA database, with log-based change data capture.
RDBMS database, with log-based change data capture supporting non-SAP underlying databases. (Oracle, SQL or dB2)
It’s important to understand the differences between these database connectors, so let me explain how we’ve developed trigger and log based CDC for SAP HANA.
SAP HANA database (Trigger)
Trigger based is our preferred method of change data capture that we recommend to SAP customers who want to replicate data out of a HANA database.
It's much easier to manage. It's very thorough. It's low touch and we have many customers that use it at volume across many sectors such as Telecommunications, Food and Beverages, Manufacturing, Healthcare and Oil & Gas.
We've made a lot of enhancements in this endpoint over the past couple of years, most of them have been in response to our customer’s needs, listening to their recommendations and by resolving their support issues.
The trigger-based endpoint is well established and hardened, it’s tried and tested with a very performant change data capture.
SAP HANA database (Log)
However, SAP customers tend to look at log-based CDC for HANA databases as they consider it to be the fastest way to replicate their data. But be aware the log-based change data capture from SAP HANA has some inherent challenges to consider, and this is not due to any limitations that we have, these are limitations that exist within the SAP HANA architecture itself.
SAP have published a note that that recommends against using log-based CDC from HANA. Because of the way that SAP writes data to a log is very different for HANA than it is for other databases. This brings complexities when dealing with the changes from the logs. We have dealt with those complexities in this endpoint, such as the concept of a unique row ID and lack of supplemental logging, and no key fields written into the logs. But there are several limitations that that we need to thread the needle on to make sure that we can support it in an individual's customs environment.
The one thing that is unique is that the log-based CDC can be used for both enterprise licenses of HANA as well as runtime licenses of HANA.
RDBMS Database (Log)
For SAP customers yet to move to HANA they will have the data residing in a Relational Database Management System (RDBMS) such as Oracle, SQL and DB2. At Qlik we have long standing experience with log-based databases. We have supported log-based CDC from SAP ECC on RDBMS databases for nearly 10-years. All that experience is in this endpoint where we are utilizing the standard log-based reader for those underlying databases with some SAP add-ons to help identify and decipher the data in those proprietary objects.
Now let’s switch to our application layer connections, as depicted in the bottom layer of the graphic.
Application Extractor connector, for data extraction from older SAP environments
Application ODP connector, utilizing Operational Data Provisioning (ODP) for low-effort data extraction from a wide variety of SAP sources.
Application OData connector, flexible data extraction from ODP-enabled OData sources
These endpoints are for customers on a HANA runtime license, or customers who want to use SAP application based features to extract data such as CDS views or extractors.
Application Extractor
An extractor is the original method that SAP uses to move data into an SAP Business warehouse (BW) system. We can call it through a custom developed API used programs and functions delivered with a transport.
It's important to know there's no restriction on the extractors. There's no SAP note that says you can't use extractors. We can freely use extractors today and we have a lot of SAP customers that do this because they're moving away from BW to something new. This is good news if this matches your situation, because it means that we can leverage the extractors that you've always used in SAP, including custom extractors that you may have built. You can now land the data in a non-SAP warehouse, such as Snowflake or in a datalake, such as Databricks. And the data will land in a format that you are already familiar with.
Application ODP
SAP Operational Data Provisioning (ODP) is the next iteration of the classic ABAP extractor method of extraction. ODP is a framework API, and we can call it directly through an RFC. It's just more complex and comprehensive than extractors because there's more proprietary objects in SAP applications today. ODP supports most extractors, CDS views, HANA information views, and BW providers. It also allows us to support table level replication through SLT with ODP as the target.
It has become very popular with SAP customers due to supporting a wide variety of SAP sources and it’s considered fast. Many vendors have created an ODP connector – However SAP have a published note for ODP, initially ODP was classed as an undocumented, unsupported API. But more recently SAP have modified the note, changing it from unsupported to unpermitted for use by 3rd parties or customers.
This is obviously creating a lot of risk and uncertainty for SAP customers, making some more risk adverse to using ODP. This is no surprise to us, and this leads us to the next endpoint.
Application OData
SAP OData is our latest released endpoint and supports all the objects that we support with ODP.
Instead of connecting directly to the ODP API, SAP have given guidance in the note to use OData as a more compliant method. OData is essentially a wrapper. We can still call ODP, but we go through OData as a secured web service. (think of it as an https call)
Now, there is some pain and some limitations for SAP customers. There's some additional configuration that you need to make in SAP as well as some considerations with memory and pagination and other things. Because of this OData is typically not as performant as an ODP endpoint.
Much like ODP many other vendors also are working on their OData endpoint, what our customers like about our OData endpoint is not only were we one of the first to market with it, but we have also highly tuned our endpoint to work with CDC. While OData may be around 30% slower than direct ODP, our OData connector has the advantaged that it has almost the same performance as ODP when in CDC mode after full load!
In addition, SAP have further updated the ODP note in July 2024, where they are pushing SAP Datasphere as an alternative solution. This was the lead up to SAP’s most recent announcement of releasing SAP Business Data Cloud in February of 2025. However, OData is still acknowledged as the preferred way to pull data from SAP systems.
Summary
Qlik has been on this journey with SAP for a long time, we have a breadth of innovation, and we have flexible techniques and solutions to get the data out of SAP. We have deep domain knowledge and expertise with SAP to guide you on the right path. At Qlik, we thrive on empowering our customers to get so much more value from all their data, SAP and non-SAP data!
How you get access to your SAP data all boils down to your choice. The many ways we help you connect to SAP cater for different use-cases, they all have their performance trade-offs and different compliancy with SAP, and licensing considerations. Whichever method you choose, have confidence we have the solution for you today and because we are continuously improving our SAP endpoints, we always will have the solution for you.
To learn more on empowering your SAP data with Qlik
Learn how to Maximize the Value of Your SAP Data
Download our whitepaper Enhance the Value of SAP with Data Integration for Analytics