Skip to content

Commit

Permalink
docs: Add Cloud Logging dashboards for new metric features (kyma-proj…
Browse files Browse the repository at this point in the history
  • Loading branch information
a-thaler authored Oct 10, 2024
1 parent f61ede7 commit edaab6a
Show file tree
Hide file tree
Showing 5 changed files with 196 additions and 39 deletions.
10 changes: 8 additions & 2 deletions docs/user/integration/sap-cloud-logging/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -304,22 +304,28 @@ Learn how to define and import recommended alerts for SAP Cloud Logging. The fol
2. To import a monitor, use the development tools of the SAP Cloud Logging dashboard.
3. Execute `POST _plugins/_alerting/monitors`, followed by the contents of the respective JSON file.
4. Depending on the pipelines you are using, enable the some or all of the following alerts:
<!-- markdown-link-check-disable -->
| Category | File | Description |
| -- | -- | -- |
| SAP Cloud Logging | [OpenSearch cluster health](https://raw.githubusercontent.com/kyma-project/telemetry-manager/main/docs/user/integration/sap-cloud-logging/alert-health.json) | The OpenSearch cluster might become unhealthy, which is indicated by a "red" status using the [cluster health API](https://opensearch.org/docs/1.3/api-reference/cluster-api/cluster-health).|
| Kyma Telemetry Integration | [Application log ingestion](https://raw.githubusercontent.com/kyma-project/telemetry-manager/main/docs/user/integration/sap-cloud-logging/alert-app-log-ingestion.json) | The LogPipeline for shipping [application logs](#ship-logs-to-sap-cloud-logging) might lose connectivity to SAP Cloud Logging, with the effect that no application logs are ingested anymore.|
| Kyma Telemetry Integration | [Access log ingestion](https://raw.githubusercontent.com/kyma-project/telemetry-manager/main/docs/user/integration/sap-cloud-logging/alert-access-log-ingestion.json) | The LogPipeline for shipping [access logs](#ship-logs-to-sap-cloud-logging) might lose connectivity to SAP Cloud Logging, with the effect that no access logs are ingested anymore. |
| Kyma Telemetry Integration | [Trace ingestion](https://raw.githubusercontent.com/kyma-project/telemetry-manager/main/docs/user/integration/sap-cloud-logging/alert-trace-ingestion.json) | The TracePipeline for shipping [traces](#ship-distributed-traces-to-sap-cloud-logging) might lose connectivity to SAP Cloud Logging, with the effect that no traces are ingested anymore. |
| Kyma Telemetry Integration | [Metric ingestion](https://raw.githubusercontent.com/kyma-project/telemetry-manager/main/docs/user/integration/sap-cloud-logging/alert-metric-ingestion.json) | The MetricPipeline for shipping [metrics](#ship-metrics-to-sap-cloud-logging) might lose connectivity to SAP Cloud Logging, with the effect that no metrics are ingested anymore. |
| Kyma Telemetry Integration | [Kyma Telemetry Status](https://raw.githubusercontent.com/kyma-project/telemetry-manager/main/docs/user/integration/sap-cloud-logging/alert-telemetry-status.json) | The Telemetry module might report a non-ready state indicating a configuration or data flow problem. |
<!-- markdown-link-check-enable -->
5. Edit notification action: Add the `destination` and adjust the intervals and thresholds to your needs.
6. Verify that the new monitor definition is listed among the SAP Cloud Logging alerts.
## Use SAP Cloud Logging Dashboards
You can view logs, traces, and metrics in SAP Cloud Logging dashboards:
<!-- markdown-link-check-disable -->
- To view the traffic and application logs, use the SAP Cloud Logging dashboards prefixed with `Kyma_`, which are based on both kinds of log ingestion: application and access logs.
- To view distributed traces, use the OpenSearch plugin **Observability**.
- To view the the container- and Pod-related metrics collected by the MetricPipeline `runtime` input, use the dashboard **[OTel] K8s Container Metrics**.
- To view the container- and Pod-related metrics collected by the MetricPipeline `runtime` input, use the dashboard **[OTel] K8s Container Metrics**.
- To view the Node-related metrics collected by the MetricPipeline `runtime` input, manually import the file [K8s Nodes](https://raw.githubusercontent.com/kyma-project/telemetry-manager/main/docs/user/integration/sap-cloud-logging/dashboard-nodes.ndjson).
- To view the status of the SAP Cloud Logging integration with the Kyma Telemetry module, manually import the file [Kyma Telemetry Status](https://raw.githubusercontent.com/kyma-project/telemetry-manager/main/docs/user/integration/sap-cloud-logging/dashboard-status.ndjson).
- To use the dashboard for Istio metrics of Pods that have an active Istio sidecar injection (collected by the MetricPipeline `istio` input), manually import the file [Kyma Istio Service Metrics](https://raw.githubusercontent.com/kyma-project/telemetry-manager/main/docs/user/integration/sap-cloud-logging/dashboard-istio.ndjson).
<!-- markdown-link-check-enable -->
152 changes: 152 additions & 0 deletions docs/user/integration/sap-cloud-logging/alert-telemetry-status.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
{
"name": "Kyma Telemetry Status",
"type": "monitor",
"monitor_type": "query_level_monitor",
"enabled": true,
"schedule": {
"period": {
"unit": "MINUTES",
"interval": 1
}
},
"inputs": [
{
"search": {
"indices": [
"metrics-otel-v1-*"
],
"query": {
"size": 0,
"query": {
"bool": {
"filter": [
{
"range": {
"time": {
"from": "{{period_end}}||-10m",
"to": "{{period_end}}",
"include_lower": true,
"include_upper": true,
"format": "epoch_millis",
"boost": 1
}
}
},
{
"match_phrase": {
"name": {
"query": "kyma.resource.status.state",
"slop": 0,
"zero_terms_query": "NONE",
"boost": 1
}
}
}
],
"must_not": [
{
"match_phrase": {
"metric.attributes.state": {
"query": "Ready",
"slop": 0,
"zero_terms_query": "NONE",
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
}
}
}
}
],
"triggers": [
{
"query_level_trigger": {
"id": "r88m6pABs6tXO7H26USv",
"name": "Inform about module problem",
"severity": "1",
"condition": {
"script": {
"source": "ctx.results[0].hits.total.value > 0",
"lang": "painless"
}
},
"actions": [
{
"id": "sM8m6pABs6tXO7H26USv",
"name": "Inform about module problem",
"destination_id": "",
"message_template": {
"source": "Monitor {{ctx.monitor.name}} just entered alert status. Please investigate the issue.\n - Trigger: {{ctx.trigger.name}}\n - Severity: {{ctx.trigger.severity}}\n - Period start: {{ctx.periodStart}}\n - Period end: {{ctx.periodEnd}}\n - https://dashboards-sf-1502277b-af0e-44ef-b169-a4d004105616.cls-01.cloud.logs.services.sap.hana.ondemand.com/app/alerting",
"lang": "mustache"
},
"throttle_enabled": true,
"subject_template": {
"source": "Module in non-ready state",
"lang": "mustache"
},
"throttle": {
"value": 60,
"unit": "MINUTES"
}
}
]
}
}
],
"ui_metadata": {
"schedule": {
"timezone": null,
"frequency": "interval",
"period": {
"unit": "MINUTES",
"interval": 1
},
"daily": 0,
"weekly": {
"tue": false,
"wed": false,
"thur": false,
"sat": false,
"fri": false,
"mon": false,
"sun": false
},
"monthly": {
"type": "day",
"day": 1
},
"cronExpression": "0 */1 * * *"
},
"search": {
"searchType": "query",
"timeField": "@timestamp",
"aggregations": [
{
"aggregationType": "max",
"fieldName": "audit_compliance_doc_version"
}
],
"groupBy": [],
"bucketValue": 1,
"bucketUnitOfTime": "h",
"where": {
"fieldName": [
{
"label": "audit_format_version",
"type": "number"
}
],
"fieldRangeEnd": 0,
"fieldRangeStart": 0,
"fieldValue": 4,
"operator": "is"
}
},
"monitor_type": "query_level_monitor"
}
}
Loading

0 comments on commit edaab6a

Please sign in to comment.