Blog > SAP CPI – Combine in Sequence vs Combine Aggregation Algorithm

SAP CPI – Combine in Sequence vs Combine Aggregation Algorithm

Mateusz_Nowak_SAP_CPI_Aggreagation
Mateusz Nowak SAP Integration Consultant, SAP Press Author
icon__calendar 2020-04-16

In this article you will learn:

  • How to use and configure from scratch Aggregator pattern in SAP CPI
  • What is the difference in practice between Aggregation Algorithm Combine and Combine in Sequence
  • How to monitor your integration flow with Aggregator pattern

Introduction

In this blog, I will guide you on how to create a sample integration flow in SAP CPI, that uses the Aggregator pattern. The aim of this blog is to show you how different aggregation algorithms work (combine vs. combine in sequence ). In our sample flow, we will use the HTTPS adapter type for inbound communication. We will send some test messages from Postman to our SAP CPI tenant, then we will use two types of Aggregation Algorithm in Aggregator pattern and in the end, we will store the log to be able to compare differences.
I hope that you will find some useful hints in this article, that you could then use in the integration battlefield!

 

SAP CPI flow with Aggregator pattern in practice

General integration flow configuration

Let’s start with the creation of the Integration Flow. Go to the Design tab in your SAP CPI tenant, open your package and create a new Integration Flow as shown below.

SAP CPI Integration flow creation

 

First of all, define inbound communication by connecting the Sender with the Start event. For the purpose of this example, please use the HTTPS adapter type.

HTTPS adapter type

 

In the details of the Connection tab provide /AggregationAlgorithmComparison as the address, UserRole as Authorization and ESBMessaging.send as a UserRole.

HTTPS adapter connection details

 

Then from the Message Routing tab, choose Aggregator pattern and place it between Start and End events. Change the name of the Aggregator to Aggregate Messages in the General tab. We will come back to the Aggregator configuration later.

Aggregator Pattern in SAP CPI

 

Now from the Message Transformers tab choose Scripts and then choose Groovy Script. Place it between Aggregator and End Event.

Groovy Script SAP CPI

 

Click on the Groovy Script that you added, change the name to Store Payload and add a new script by clicking on the Create (+) icon. Copy the following lines of code from code snippet to the script and then click OK.

 

Alternatively, you could use Trace mode for integration flow to see the payload, but for the purpose of the example let’s store it using Groovy Script – it will allow you to have quicker access to the payload. 

Save the integration flow. So far it should look as shown below:

Aggregator pattern in the Integration Flow

 

Aggregator Pattern Configuration

Now it is time for aggregator pattern configuration. For both aggregation algorithms, we will send exactly the same 6 messages in exactly the same order. Thanks to this we will be able to compare the results after processing both algorithms. 

We will send 6 messages with the log from the cashes from 2 stores: Sport ABC and Matt. Our goal is to aggregate all messages that are being sent in the time frame of 2 minutes based on Shop Name.

time frame table SAP CPI

 

Based on the example messages shown in the table above we would expect to receive 2 messages with all transactions per specific store considering the fact that the time between receiving those messages doesn’t exceed 2 minutes.

Below you can see a code snippet that contains all 6 XML messages that we will send after configuration of the Aggregator step.

 

Combine in Sequence Aggregation Algorithm

First, we will design the flow for the scenario, where messages should be aggregated based on the ShopName, in the sequence determined by the TransactionID. As mentioned before, the maximum time between 2 messages should not exceed 2 minutes for the aggregation.  


Considering all the conditions above, in the end, we should have two messages with the following data:

 

SAP CPI table Aggregator

 

Ok, let’s have some fun configuring such a scenario!

Click on the Aggregate Messages step and go to the Correlation Tab. In the Correlation Expression (XPath) we have to define the path to the node, based on which we want to aggregate messages. It means that all the messages with the same value in this particular node will be merged into one message. In our example this is Shop Name, so use the following Xpath: /ShoppingLog/ShopName.

Aggregation Strategy

Next, go to the Aggregation Strategy tab. Here we have to provide:

 

  • Incoming Format: XML (Same Format)
    This is the only option that currently can be chosen. Note: Aggregator pattern supports only XMLs that are in the same format.
  • Aggregation Algorithm: Combine in Sequence
    All the messages that will be received in the timeframe of 2 minutes will be first sorted based on the message sequence expression and only afterward they will be processed.
  • Message Sequence Expression (XPath): /ShoppingLog/TransactionID
    Contains XPath to the node based on which sequence is defined. What is important in this step, is the fact that the Sequence number cannot be larger than 100000. Considering this, it can be quite hard to use for example, timestamp field for message sequence expression.
  • Last Message Condition (XPath): /ShoppingLog/LastOrder=’X’
    Contains XPath to the node based on which Aggregator finishes the current aggregation. For example, if the message with value ‘X’ will be sent in the LastOrder node, the next message that will be sent even for the same ShopName within 2 minutes will not be aggregated with the previous messages.
  • Completion Timeout (in min): 2
  • Data Store Name: AggregatorExample
    Name of your local data store.

Combine in Sequence Aggregation Algorithm details

 

Well done! You have just configured the Aggregator pattern with Combine in Sequence algorithm. Save your integration flow and deploy it.

Now go to the Operations View and then to the Manage Integration Content to make sure that your integration flow has started successfully. Then copy the endpoint URL that we will use to send messages from the Postman.

Successfully deployed integration flow

 

In the Postman paste the URL to your integration flow endpoint, provide Basic Authentication to your SAP CPI tenant with authorization for ESBMessage.send, go to the Body tap, choose raw data format and paste the first XML message and then click Send.

Postman request with sample message

 

Send all 6 messages one by one from the code snippet above.

Now you can go to the SAP CPI Operations View in order to see processed messages in the Monitor Message processing. Ideally, you should see 8 messages for your integration flow. You can see 6 messages you sent from the Postman and two additional messages (2nd and 5th) that are still processing. Those 6 messages are the ones that have been received to the Aggregator and the other 2 that contain the aggregated messages in the data store. 

SAP CPI Monitor Message Processing - Aggregation Algorithm Combine in Sequence

 

Once the messages with aggregated messages are completed you can go into details, where you should see an extended log for runs, that consist of 4 messages.  

 

SAP CPI - Aggregation Algorithm Runs

 

In order to make sure that our messages have been correctly aggregated, let’s see logs with the payload for both of them.

Log for the Shop Name: Matt. As you can see the output of the aggregation is wrapped in <multimap:Message1> node.

Payload for aggregated in sequence messages for shop Matt in SAP CPI

  • The message contains only the messages with ShopName Matt – correct 🙂
  • Originally messages were sent in the following order by transaction ID: 320, 300, 310. In log we can see that messages were sorted based on Transaction ID and in the log we can see them in the correct order- 300, 310, 320 – correct 🙂

Log for the Shop Name: Sport ABC.

Payload for aggregated in sequence messages for shop Matt in SAP CPI

  • The message contains only the messages with ShopName Sport ABC – correct 🙂
  • Originally messages were sent in the following order by transaction ID: 555, 554, 553. In the log we can see that messages were sorted based on Transaction ID and in the log we can see them in the correct order- 553, 554, 555 – correct 🙂

 

Nicely done! Your first scenario with Combine in Sequence Aggregation Algorithm is working as expected. 

 

Combine Aggregation Algorithm

In order to compare the Combine in Sequence vs. Combine aggregation algorithm, let’s go back to your integration flow, jump to the Aggregate Messages step and go to the Aggregation Strategy tab. Change the Aggregation Algorithm from Combine in Sequence to Combine. The field Message Sequence Expression (XPath) should disappear as shown below.

Combine Aggregation Algorithm details

 

Save your integration flow, deploy and once it’s successfully Started to go to the Postman and send all six messages in exactly the same order as previous. 

What do we expect in the results in your opinion? Will we have again 2 messages that contain 3 separate messages merged?

I think you already guessed that this is not true. Why is that? Before answering this question, let’s look at our message in the Monitor for processed messages. We will have more messages at the first glance in comparison to a combined sequence algorithm, as shown below. Instead of 8 messages – we will have 10 messages.

SAP CPI Monitor Message Processing - Aggregation Algorithm Combine

 

Again, we will have six messages that have been directly sent via the Postman, but instead of 2 aggregated messages, we will see 4 messages in SAP CPI Monitor. This is because the first messages that you sent contained also the indicator for the last message – value ‘X’ in the LastOrder. Since SAP CPI aggregation Combine was not in sequence – messages have been processed in the order they were sent. 

 

As a proof, we can see below code snippet containing 4 payloads from the aggregated messages. Each payload is aggregated between <multimap:Messages> node.

 

 

SAP CPI Combine in Sequence vs Combine Aggregation Algorithms summary

Congratulations! If you are reading this, it means that you went through both aggregation algorithms and you are aware of the major differences between them. You have configured the scenario, where all messages from specific ShopName that have been received within a 2 minutes time frame should be mapped together. Additionally, you used the last message condition in LastOrder, which allowed you to determine when specific aggregation should be finished and you were able to see how it affected the aggregation when the sequence is used or not. You can call yourself the SAP CPI Aggregator Ninja now! 🙂

I also truly recommend you to visit SAP Help for the Aggregator pattern. You can also find more blogs about SAP Integration and SAP CPI on our blog, for example here you can find amazing Andrzej’s blog where he shows how to control messages with a custom header.

If you are reading this before the 30th of April, you still have the opportunity to get one of our SAP Press books – just take part in the SAP CPI challenge prepared by Eng Swee Yeoh. 

Do you use Aggregator quite often in your scenarios? What aggregation algorithms do you use then? Let us know in the comments!

 

Mateusz Nowak SAP Integration Consultant, SAP Press Author
SAP Integration consultant since 2016. Mateusz is the author of SAP Press Book: Interface Monitoring and Error Handling with SAP AIF. He's also a trainer in the openSAP course Virtualize and Automate Your SAP Testing Using Int4 IFTT. Matt has been involved in multiple integrational projects as ABAP Developer, AIF lead and consultant. Skilled at a variety of tools and development techniques including ABAP, AIF, PO, BRFplus, IDocs, Adobe Forms, Smartforms, IDocs, ALE, SCPI.