Skip to content

Commit

Permalink
Update from SAP DITA CMS (squashed):
Browse files Browse the repository at this point in the history
commit ef91fbe7b6ba173f446c2d02e0b3c7a951d31416
Author: REDACTED
Date:   Wed Jul 10 12:51:40 2024 +0000

    Update from SAP DITA CMS 2024-07-10 12:51:40
    Project: dita-all/rqs1627049557344
    Project map: eeb0d7e6335f4e73b4158bf8f2d82b8c.ditamap
    Output: loio2fb418ef35ea47f9ad0d99b3ed84b370
    Language: en-US
    Builddable map: 11b78b7d7def4d739976a99060a8b676.ditamap

commit c92008848f8a244d3d390622e5b3497f43446bce
Author: REDACTED
Date:   Wed Jul 10 08:58:09 2024 +0000

    Update from SAP DITA CMS 2024-07-10 08:58:09
    Project: dita-all/rqs1627049557344
    Project map: eeb0d7e6335f4e73b4158bf8f2d82b8c.ditamap
    Output: loio2fb418ef35ea47f9ad0d99b3ed84b370
    Language: en-US
    Builddable map: 11b78b7d7def4d739976a99060a8b676.ditamap

commit b253eef3284f0e6cce0297f99135bee58c794de8
Author: REDACTED
Date:   Wed Jun 26 08:36:51 2024 +0000

    Update from SAP DITA CMS 2024-06-26 08:36:51
    Project: dita-all/rqs1627049557344
    Project map: eeb0d7e6335f4e73b4158bf8f2d82b8c.ditamap
    Output: loio2fb418ef35ea47f9ad0d99b3ed84b370
    Language: en-US
    Builddable map: 11b78b7d7def4d739976a99060a8b676.ditamap

##################################################
[Remaining squash message was removed before commit...]
  • Loading branch information
ditaccms-bot committed Jul 10, 2024
1 parent 2424da6 commit 0822af7
Show file tree
Hide file tree
Showing 13 changed files with 376 additions and 40 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ The following measures improve monitoring and error handling in the pipeline con
- Use **SAP headers** to search for messages based on sender system, interface/message type, and receiver system.

- Set a **custom status** for more information on the error root cause.
- Use **retry handling** to limit the number of overall retries. See [Retry Handling](monitoring-and-error-handling-in-the-pipeline-concept-ed9b82c.md#loioed9b82cb928049e6990a4d784aa6aac7__section_l3k_qrn_j1c).
- Use **retry handling** to limit the number of overall retries. See [Standard Retry Handling](monitoring-and-error-handling-in-the-pipeline-concept-ed9b82c.md#loioed9b82cb928049e6990a4d784aa6aac7__section_l3k_qrn_j1c).

- User **custom exception handling** as an alternative to the standard retry handling and implement your own logic. See [Custom Exception Handling](monitoring-and-error-handling-in-the-pipeline-concept-ed9b82c.md#loioed9b82cb928049e6990a4d784aa6aac7__section_pm1_ggs_5bc).
- With a **dead letter queue**, you can park messages for which the maximum number of retries has been exceeded. See [Dead Letter Queue](monitoring-and-error-handling-in-the-pipeline-concept-ed9b82c.md#loioed9b82cb928049e6990a4d784aa6aac7__section_lq1_2sn_j1c).

- Attach **additional information to the message processing log** if errors occur.
Expand All @@ -22,34 +23,216 @@ The following measures improve monitoring and error handling in the pipeline con

<a name="loioed9b82cb928049e6990a4d784aa6aac7__section_l3k_qrn_j1c"/>

## Retry Handling
## Standard Retry Handling

If you want to define a scenario-specific retry behavior, you need to create a string parameter with ID `MaxJMSRetries`. Otherwise, the maximum number of retries is set to unlimited. In the generic inbound processing integration flow, this parameter is read from the Partner Directory and written in the message header `maxJMSRetries`. The header is passed to all other integration flows in the sequence of flows.

In all integration flows that read from a JMS queue, the following retry handling is implemented.
In all integration flows that read from a JMS queue, the following standard retry handling is implemented. For custom retry handling, see [Custom Exception Handling](monitoring-and-error-handling-in-the-pipeline-concept-ed9b82c.md#loioed9b82cb928049e6990a4d784aa6aac7__section_pm1_ggs_5bc).

If a message processing error occurs, the error is fetched in an exception subprocess. Within the exception subprocess, a local integration process is called. Then, the local integration process checks if the maximum number of retries has already been exceeded using the following condition expression:

```
${header.SAPJMSRetries} > ${header.maxJMSRetries} and ${header.maxJMSRetries} != 'unlimited'
```

The default route, that is, if the maximum number of retries hasn't been exceeded, ends with an **error end event**. This means that the message remains in the status *Failed* and is retried from the inbound JMS queue.
The default route, that is, if the maximum number of retries hasn't been exceeded, sets the custom status to *AutomaticRetry* and then ends with an **error end event**. This means that the message remains in the status *Failed* and is retried from the inbound JMS queue.

Otherwise, if the maximum number of retries has been exceeded, the message ends with an **end event**. Then, the subsequent integration flow steps within the exception subprocess are carried out.

In a content modifier step, the custom status is set to `MaxRetriesExceeded`. In a Groovy script, additional information about the inbound queue name and the dead letter queue name is attached to the message processing log. The name of the dead letter queue follows the following pattern:
Otherwise, if the maximum number of retries has been exceeded, the custom status is set to `MaxRetriesExceeded`. In a Groovy script, additional information about the inbound queue name and the dead letter queue name is attached to the message processing log. The name of the dead letter queue follows the following pattern:

`<name of the inbound queue>_DLQ`

The message is then written to the dead letter queue and the message status is set to *Successful*.
The message is then written to the dead letter queue and ends with an end event, which then sets the message status to *Successful*.

The following is an example of retry handling:

![The screenshot shows the integration flow editor with an example of retry handling using an exception subprocess and a dead letter queue.](images/PipelineConcept_RetryHandling_99dcda0.png)



<a name="loioed9b82cb928049e6990a4d784aa6aac7__section_pm1_ggs_5bc"/>

## Custom Exception Handling

As an alternative to the standard retry handling, you can implement your own custom exception handling. The generic integration flows support a kind of custom exit that you can use to run your own custom exception logic without changing the delivered generic integration flows. To do so, you must implement a separate integration flow that's called instead of the standard retry handling if an exception occurs.

To enable the custom exception handling, configure the generic integration flows as follows:

1. In the integration flow configuration, switch to the tab *More* and set the parameter *CustomXError\_Enabled* to `true` \(the default value is `false`\).

2. Next, in the tab *Receiver*, select the receiver *CustomErrorFlow* of adapter type *ProcessDirect*. Maintain the end point of the integration flow that should be called in an exception occurs. By default, the value is set to `/pip/custom/errorHandling`. You can keep that value or change it to your preference.

Instead of the standard retry handling, the local integration process `Call custom error handling` is now called.

3. You can exchange message headers and the message body between the generic integration flows and the custom exception handling integration flow.

In a content modifier step, the following headers are passed to the custom exception handling integration flow:


<table>
<tr>
<th valign="top">

Name

</th>
<th valign="top">

Source Type

</th>
<th valign="top">

Source Value

</th>
</tr>
<tr>
<td valign="top">

`exceptionTimestamp`

</td>
<td valign="top">

Expression

</td>
<td valign="top">

`${date:now:dd-MM-yyyy HH:mm z}`

</td>
</tr>
<tr>
<td valign="top">

`exceptionStacktrace`

</td>
<td valign="top">

Expression

</td>
<td valign="top">

`${exception.stacktrace`

</td>
</tr>
<tr>
<td valign="top">

`exceptionMessage`

</td>
<td valign="top">

Expression

</td>
<td valign="top">

`${exception.message}`

</td>
</tr>
<tr>
<td valign="top">

`pipelineStepID`

</td>
<td valign="top">

Expression

</td>
<td valign="top">

`${camelId}`

</td>
</tr>
<tr>
<td valign="top">

`partnerID`

</td>
<td valign="top">

Expression

</td>
<td valign="top">

`${property.partnerID}`

</td>
</tr>
<tr>
<td valign="top">

`incomingQueue`

</td>
<td valign="top">

Expression

</td>
<td valign="top">

`{{JMS_Incoming}}`

</td>
</tr>
</table>

Additionally, the following further headers of the Camel runtime or headers that have already been defined in the generic integration flow before are also passed to the custom exception handling integration flow:

- `SAP_Sender`

- `SAP_SenderInterface`

- `SAP_MessageType`

- `SAP_Receiver`

- `maxJMSRetries`

- `SAPJMSRetries`

- `testMode`


You can use these headers within the custom exception handling integration flow to implement your custom exception logic. If you do so, remember to enter the headers in the *Allowed Header\(s\)* runtime configuration in the custom exception handling integration flow.

4. In a request reply step, the custom exception handling integration flow is then called via a ProcessDirect adapter.

5. As response, the headers *customXError\_Status* and *customXError\_EndEvent* are excepted. The headers must be set in the custom exception handling integration flow.

6. In a content modifier, the custom status is set to the value of the header *customXError\_Status*.

7. Depending on the value of the *customXError\_EndEvent*, the local integration process ends as follows:

- If the value equals *Complete*, it ends with an end event.

- If the value equals *Escalated*, it ends with an escalation end event.

- In all other cases, it ends with an error end event.



The following screenshot shows the local integration flow as it calls the custom exception handling integration flow:

![](images/Call_custom_error_handling_a64762d.png)

You can either create your custom exception handling flow from scratch or use the provided integration flow `Pipeline Template – Custom Error Handling` as template. The `Pipeline Template - Custom Error Handling` has implemented a similar logic like the standard retry handling. If you define your own integration flow from scratch, ensure that the allowed headers are set as described in the previous steps. You also need to pass back the headers *customXError\_Status* and *customXError\_EndEvent*.



<a name="loioed9b82cb928049e6990a4d784aa6aac7__section_xfy_1sn_j1c"/>

## Message Monitoring
Expand All @@ -58,7 +241,7 @@ To showcase the monitoring handling, the following section discusses the monitor

In the SAP Integration Suite, go to *Monitor* \> *Integration and APIs* \> *Monitor Message Processing*. Here, you can filter for the **correlation ID**, which shows the logs of the different pipeline steps that have been carried out so far for a particular message. You can also see that Sender and Application Message Type are set as well, so you can also filter for the sender system and the sender interface name.

If the inbound conversion failed, the logs of the scenario-specific conversion integration flow are in status *Failed*. In this example, you see two failed logs because one restart ha already been carried out. The custom status is set to *RetryViaParentFlow*, which indicates that the retry is handled by the calling integration flow.
If the inbound conversion failed, the logs of the scenario-specific conversion integration flow are in status *Failed*. In this example, you see two failed logs because one restart has already been carried out. The custom status is set to *RetryViaParentFlow*, which indicates that the retry is handled by the calling integration flow.

The following is an example of a failed inbound conversion integration flow:

Expand Down
3 changes: 3 additions & 0 deletions docs/70-interface-migration/pipeline-concept-6e527fb.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,4 +109,7 @@ Perform monitoring and error handling in the pipeline concept, for example by us
- **[Customizing the Pipeline Concept](customizing-the-pipeline-concept-aeb106f.md "You can apply the pipeline concept as a template and fit it to your custom needs. For example, implement your own exception handling, or
define a different global maximum number of retries.")**
You can apply the pipeline concept as a template and fit it to your custom needs. For example, implement your own exception handling, or define a different global maximum number of retries.
- **[Testing the Pipeline](testing-the-pipeline-e373569.md "SAP's partners provide regression test tools that you can use to test your integration scenarios and apply the pipeline
concept.")**
SAP's partners provide regression test tools that you can use to test your integration scenarios and apply the pipeline concept.

2 changes: 1 addition & 1 deletion docs/70-interface-migration/pipeline-steps-f8e69f4.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ If no conversion is needed, the message is passed through and directly stored in
If a conversion error occurs, the error is fetched in an exception subprocess, which first checks if the maximum number of retries have been exceeded. If they haven't been exceeded, the message remains in the first JMS queue from which it's retried. Otherwise, the message is parked in a so-called dead letter queue.

> ### Note:
> You can define the maximum number of retries for a specific scenario. This is also stored in the Partner Directory. See [Retry Handling](monitoring-and-error-handling-in-the-pipeline-concept-ed9b82c.md#loioed9b82cb928049e6990a4d784aa6aac7__section_l3k_qrn_j1c).
> You can define the maximum number of retries for a specific scenario. This is also stored in the Partner Directory. See [Standard Retry Handling](monitoring-and-error-handling-in-the-pipeline-concept-ed9b82c.md#loioed9b82cb928049e6990a4d784aa6aac7__section_l3k_qrn_j1c).
The following screenshot is an example of a generic integration flow for inbound processing :

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,7 @@ To import or migrate import mapping artifacts from SAP Process Orchestration to
However, local Java UDFs are supported and can be viewed and edited.

> ### Note:
> Lookups \(for example, RFC, JDBC\) are not supported. Use additional integration flow steps instead. For instructions, see the tutorial [Learn how to migrate JDBC Lookups from SAP Process Orchestration to Cloud Integration](https://developers.sap.com/tutorials/ci-jdbc-lookup.html).
> Lookups \(for example, RFC, JDBC\) are not supported. Use additional integration flow steps instead. For instructions, see the tutorials [Learn how to migrate JDBC Lookups from SAP Process Orchestration to Cloud Integration](https://developers.sap.com/tutorials/ci-jdbc-lookup.html) and [How to migrate RFC Lookups from SAP Process Orchestration to Cloud Integration](https://developers.sap.com/tutorials/ci-rfclookup.html).


Expand Down Expand Up @@ -363,7 +363,7 @@ To import or migrate import mapping artifacts from SAP Process Orchestration to
Message mapping containing reference to function libraries is not supported. You need to redesign these mappings, using Javascript/Groovy script. The scripts can be created locally in the integration flow or you can create a script collection to make it available for different integration flows inside your package. See the blog [Script collection reusable artifact in SAP Cloud Integration](https://blogs.sap.com/2021/06/07/script-reusable-artifact-in-sap-cloud-integration/).

> ### Note:
> Lookups \(for example, RFC, JDBC\) are not supported. Use additional integration flow steps instead. For instructions, see the tutorial [Learn how to migrate JDBC Lookups from SAP Process Orchestration to Cloud Integration](https://developers.sap.com/tutorials/ci-jdbc-lookup.html).
> Lookups \(for example, RFC, JDBC\) are not supported. Use additional integration flow steps instead. For instructions, see the tutorials [Learn how to migrate JDBC Lookups from SAP Process Orchestration to Cloud Integration](https://developers.sap.com/tutorials/ci-jdbc-lookup.html) and [How to migrate RFC Lookups from SAP Process Orchestration to Cloud Integration](https://developers.sap.com/tutorials/ci-rfclookup.html).


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,34 +18,39 @@ The following scripts are part of the script collection `Pipeline Generic - Scri

- [`readReceiverNotDeterminedFromPD`](script-collection-for-pipeline-concept-05d9f8d.md#loio05d9f8d85d7945af8d610dca81375b43__section_pmr_my2_j1c)

- <code><a href="script-collection-for-pipeline-concept-05d9f8d.md#loio05d9f8d85d7945af8d610dca81375b43__section_yjh_bh4_vbc">displayBulkIDs</a></code>
- <code><a href="script-collection-for-pipeline-concept-05d9f8d.md#loio05d9f8d85d7945af8d610dca81375b43__section_dp4_xj4_vbc">setIDocSenderInterface</a></code>



<a name="loio05d9f8d85d7945af8d610dca81375b43__section_qdh_px2_j1c"/>

## attachAdditionalInformation

Use the following Groovy script `attachAdditionalInformation` to add information about the queue names to the message processing log if the message is parked in a dead letter queue.
Use the following Groovy script `attachAdditionalInformation` to add information about the queue names to the message processing log if the message is parked in a dead letter queue. In addition, the log level is set to `DEBUG`.

```
import com.sap.gateway.ip.core.customdev.util.Message
import java.util.HashMap
def Message processData(Message message) {
// get queue names
def propertyMap = message.getProperties()
def incomingQueue = propertyMap.get("incomingQueue");
def deadLetterQueue = propertyMap.get("deadLetterQueue");
def messageLog = messageLogFactory.getMessageLog(message);
if (messageLog != null) {
body = 'Maximum number of retries exceeded'
body = body + '\nMessage removed from queue ' + incomingQueue + ' and sent to dead letter queue ' + deadLetterQueue;
messageLog.addAttachmentAsString('Additional Information', body, 'text/plain');
}
return message;
}
import com.sap.gateway.ip.core.customdev.util.Message
import java.util.HashMap
def Message processData(Message message) {
// get queue names
def propertyMap = message.getProperties()
def incomingQueue = propertyMap.get("incomingQueue");
def deadLetterQueue = incomingQueue + '_DLQ';
def messageLog = messageLogFactory.getMessageLog(message);
if (messageLog != null) {
def body = 'Maximum number of retries exceeded'
body = body + '\nMessage removed from queue ' + incomingQueue + ' and sent to dead letter queue ' + deadLetterQueue;
// add attachment to message processing log
messageLog.addAttachmentAsString('Additional Information', body, 'text/plain');
// set log level to DEBUG at message level
message.setHeader('SAP_MessageProcessingLogLevel', 'DEBUG');
}
return message;
}
```


Expand Down Expand Up @@ -262,3 +267,51 @@ def Message processData(Message message) {
}
```



<a name="loio05d9f8d85d7945af8d610dca81375b43__section_yjh_bh4_vbc"/>

## displayBulkIDs

Use the following Groovy script `displayBulkIDs` to display all IDoc numbers of an IDoc bulk message in the message monitor.

```
import com.sap.gateway.ip.core.customdev.util.Message
import org.w3c.dom.NodeList
def Message processData(Message message) {
def nodes = message.getProperty('nodelistToHeader') as NodeList
def headerName = message.getProperty('nodelistName') as String
def messageLog = messageLogFactory.getMessageLog(message)
for (i = 0; i < nodes.getLength(); i++) {
messageLog?.addCustomHeaderProperty(headerName, nodes.item(i).getTextContent())
}
return message
}
```



<a name="loio05d9f8d85d7945af8d610dca81375b43__section_dp4_xj4_vbc"/>

## setIDocSenderInterface

Use the following Groovy script `setIDocSenderInterface` to concatenate the sender interface based on the IDoc control headers `MESTYP`, `IDOCTYP` and `CIMTYP`.

```
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
def Message processData(Message message) {
// headers
def map = message.getHeaders()
// set sender interface name
message.setHeader("SAP_SenderInterface", map.get("SAP_IDoc_EDIDC_MESTYP") + "." + map.get("SAP_IDoc_EDIDC_IDOCTYP") + (map.get("SAP_IDoc_EDIDC_CIMTYP") ? "." +map.get("SAP_IDoc_EDIDC_CIMTYP") : ""))
return message;
}
```

Loading

0 comments on commit 0822af7

Please sign in to comment.