Skip to content

Commit

Permalink
docs: Add docsify Flexible Alerts plugin for Telemetry (kyma-project#792
Browse files Browse the repository at this point in the history
)
  • Loading branch information
NHingerl authored Feb 14, 2024
1 parent 1f00300 commit d1348ef
Show file tree
Hide file tree
Showing 9 changed files with 56 additions and 27 deletions.
9 changes: 6 additions & 3 deletions docs/user/02-logs.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,8 @@ An output is a data destination configured by a [Fluent Bit output](https://docs
- **http**, which sends the data to the specified HTTP destination. The output is designed to integrate with a [Fluentd HTTP Input](https://docs.fluentd.org/input/http), which opens up a huge ecosystem of integration possibilities.
- **custom**, which supports the configuration of any destination in the Fluent Bit configuration syntax.
>**CAUTION:** If you use a `custom` output, you put the LogPipeline in the [unsupported mode](#unsupported-mode).
> [!WARNING]
> If you use a `custom` output, you put the LogPipeline in the [unsupported mode](#unsupported-mode).

See the following example of the `custom` output:

Expand Down Expand Up @@ -160,7 +161,8 @@ spec:
...
```

> **CAUTION:** If you use a `custom` output, you put the LogPipeline in the [unsupported mode](#unsupported-mode).
> [!WARNING]
> If you use a `custom` output, you put the LogPipeline in the [unsupported mode](#unsupported-mode).

Telemetry Manager supports different types of [Fluent Bit filter](https://docs.fluentbit.io/manual/concepts/data-pipeline/filter). The example uses the filters [grep](https://docs.fluentbit.io/manual/pipeline/filters/grep) and [record_modifier](https://docs.fluentbit.io/manual/pipeline/filters/record-modifier).

Expand Down Expand Up @@ -296,7 +298,8 @@ spec:
...
```

> **NOTE:** If you use a `custom` output, you put the LogPipeline in the [unsupported mode](#unsupported-mode).
> [!NOTE]
> If you use a `custom` output, you put the LogPipeline in the [unsupported mode](#unsupported-mode).

### Step 4: Rotate the Secret

Expand Down
11 changes: 7 additions & 4 deletions docs/user/03-traces.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ spec:

### Step 2. Enable Istio Tracing

>**CAUTION:** The provided Istio feature uses an API in alpha state, which may change in future releases.
> [!WARNING]
> The provided Istio feature uses an API in alpha state, which may change in future releases.

By default, the tracing feature of the Istio module is disabled to avoid increased network utilization if there is no TracePipeline.
To activate the Istio tracing feature with a sampling rate of 5% (for recommendations, see [Istio](#istio)), use a resource similar to the following:
Expand Down Expand Up @@ -314,7 +315,8 @@ Kyma bundles several modules which are potentially involved in user flows. Appli

The Istio module is crucial in distributed tracing because it provides the [ingress gateway](https://istio.io/latest/docs/tasks/traffic-management/ingress/ingress-control/). Usually, this is where external requests enter the cluster scope and are enriched with trace context if it hasn't happened yet. Furthermore, every component that's part of the Istio Service Mesh runs an Istio proxy, which propagates the context properly but also creates span data. If Istio tracing is activated and taking care of trace propagation in your application, you get a complete picture of a trace, because every component automatically contributes span data. Also, Istio tracing is pre-configured to be based on the vendor-neutral [w3c-tracecontext](https://www.w3.org/TR/trace-context/) protocol.

>**CAUTION:** The provided Istio feature uses an API in alpha state, which may change in future releases.
> [!WARNING]
> The provided Istio feature uses an API in alpha state, which may change in future releases.

The Istio module is configured with an [extension provider](https://istio.io/latest/docs/tasks/observability/telemetry/) called `kyma-traces`. To activate the provider on the global mesh level using the Istio [Telemetry API](https://istio.io/latest/docs/reference/config/telemetry/#Tracing), place a resource to the `istio-system` namespace. The following snippets help setting up the Istio tracing feature:

Expand All @@ -340,8 +342,9 @@ spec:

By default, the sampling rate is configured to 1%. That means that only 1 trace out of 100 traces is reported to the trace gateway, and all others are dropped. The sampling decision itself is propagated as part of the [trace context](https://www.w3.org/TR/trace-context/#sampled-flag) so that either all involved components are reporting the span data of a trace, or none.

>**TIP:** If you increase the sampling rate, you send more data your tracing backend and cause much higher network utilization in the cluster.
>To reduce costs and performance impacts in a production setup, a very low percentage of around 5% is recommended.
> [!TIP]
> If you increase the sampling rate, you send more data your tracing backend and cause much higher network utilization in the cluster.
> To reduce costs and performance impacts in a production setup, a very low percentage of around 5% is recommended.

To configure an "always-on" sampling, set the sampling rate to 100%:

Expand Down
6 changes: 4 additions & 2 deletions docs/user/04-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,8 @@ If you use a Secret owned by the [SAP BTP Service Operator](https://github.com/S

### Step 4: Activate Prometheus-Based Metrics

> **NOTE:** For the following approach, you must have instrumented your application using a library like the [Prometheus client library](https://prometheus.io/docs/instrumenting/clientlibs/), with a port in your workload exposed serving as a Prometheus metrics endpoint.
> [!NOTE]
> For the following approach, you must have instrumented your application using a library like the [Prometheus client library](https://prometheus.io/docs/instrumenting/clientlibs/), with a port in your workload exposed serving as a Prometheus metrics endpoint.

To enable collection of Prometheus-based metrics, define a MetricPipeline that has the `prometheus` section enabled as input:

Expand Down Expand Up @@ -307,7 +308,8 @@ Put the following annotations either to a Service that resolves your metrics por
| `prometheus.io/path` | `/metrics`, `/custom_metrics` | `/metrics` | Defines the HTTP path where Prometheus can find metrics data. |
| `prometheus.io/scheme` | `http`, `https` | If Istio is active, `https` is supported; otherwise, only `http` is available. The default scheme is `http` unless an Istio sidecar is present, denoted by the label `security.istio.io/tlsMode=istio`, in which case `https` becomes the default. | Determines the protocol used for scraping metrics — either HTTPS with mTLS or plain HTTP. |

> **NOTE:** The agent can scrape endpoints even if the workload is a part of the Istio service mesh and accepts mTLS communication. However, there's a constraint: For scraping through HTTPS, Istio must configure the workload using 'STRICT' mTLS mode. Without 'STRICT' mTLS mode, you can set up scraping through HTTP by applying the `prometheus.io/scheme=http` annotation. For related troubleshooting, see [Log entry: Failed to scrape Prometheus endpoint](#log-entry-failed-to-scrape-prometheus-endpoint).
> [!NOTE]
> The agent can scrape endpoints even if the workload is a part of the Istio service mesh and accepts mTLS communication. However, there's a constraint: For scraping through HTTPS, Istio must configure the workload using 'STRICT' mTLS mode. Without 'STRICT' mTLS mode, you can set up scraping through HTTP by applying the `prometheus.io/scheme=http` annotation. For related troubleshooting, see [Log entry: Failed to scrape Prometheus endpoint](#log-entry-failed-to-scrape-prometheus-endpoint).

### Step 5: Activate Runtime Metrics

Expand Down
3 changes: 2 additions & 1 deletion docs/user/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ Fundamentally, ["Observability"](https://opentelemetry.io/docs/concepts/observab

Kyma's Telemetry module focuses exactly on the aspects of instrumentation, collection, and shipment that happen in the runtime and explicitly defocuses on backends.

> **TIP:** An enterprise-grade setup demands a central solution outside the cluster, so we recommend in-cluster solutions only for testing purposes. If you want to install lightweight in-cluster backends for demo or development purposes, check the [Telemetry integration guides](#integration-guides).
> [!TIP]
> An enterprise-grade setup demands a central solution outside the cluster, so we recommend in-cluster solutions only for testing purposes. If you want to install lightweight in-cluster backends for demo or development purposes, check the [Telemetry integration guides](#integration-guides).
## Features

Expand Down
24 changes: 16 additions & 8 deletions docs/user/integration/aws-cloudwatch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,17 @@ Fluent Bit ingests logs directly into CloudWatch using the [Amazon CloudWatch ou

## Table of Content

- [Prerequisites](#prerequisites)
- [Prepare the Namespace](#prepare-the-namespace)
- [Set Up AWS Credentials](#set-up-aws-credentials)
- [Deploy the AWS Distro](#deploy-the-aws-distro)
- [Set Up Kyma Pipelines](#set-up-kyma-pipelines)
- [Verify the Results](#verify-the-results)
- [Integrate Kyma with Amazon CloudWatch and AWS X-Ray](#integrate-kyma-with-amazon-cloudwatch-and-aws-x-ray)
- [Overview](#overview)
- [Table of Content](#table-of-content)
- [Prerequisites](#prerequisites)
- [Prepare the Namespace](#prepare-the-namespace)
- [Set Up AWS Credentials](#set-up-aws-credentials)
- [Create AWS IAM User](#create-aws-iam-user)
- [Create a Secret with AWS Credentials](#create-a-secret-with-aws-credentials)
- [Deploy the AWS Distro](#deploy-the-aws-distro)
- [Set Up Kyma Pipelines](#set-up-kyma-pipelines)
- [Verify the Results](#verify-the-results)

## Prerequisites

Expand Down Expand Up @@ -63,7 +68,8 @@ kubectl create secret generic aws-credentials -n $K8S_NAMESPACE --from-literal=A
Deploy the AWS Distro, which is an AWS-supported distribution of an OTel collector. The [AWS X-Ray Tracing Exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/awsxrayexporter) used in the collector converts OTLP traces to [AWS X-Ray Segment Documents](https://docs.aws.amazon.com/xray/latest/devguide/xray-api-segmentdocuments.html) and then sends them directly to X-Ray. The [AWS CloudWatch EMF Exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/awsemfexporter/README.md) used in the collector converts OTLP metrics to [AWS CloudWatch Embedded Metric Format(EMF)](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html) and then sends them directly to CloudWatch Logs.
> **NOTE:** The retention of these CloudWatch Logs is set to 7 days. You can change that to fit your needs by adjusting the `log_retention` value for the `awsemf` exporter in the [`aws-otel.yaml`](aws-otel.yaml) file.
> [!NOTE]
> The retention of these CloudWatch Logs is set to 7 days. You can change that to fit your needs by adjusting the `log_retention` value for the `awsemf` exporter in the [`aws-otel.yaml`](aws-otel.yaml) file.
```bash
kubectl -n $K8S_NAMESPACE apply -f aws-otel.yaml
Expand All @@ -74,7 +80,9 @@ kubectl -n $K8S_NAMESPACE apply -f aws-otel.yaml
Use the Kyma Telemetry module to enable ingestion of the signals from your workloads:
1. Deploy a LogPipeline:
> **NOTE:** The retention of of the CloudWatch Logs is set to 7 days. You can change that to fit your needs by adjusting the `log_retention_days` value.
> [!NOTE]
> The retention of of the CloudWatch Logs is set to 7 days. You can change that to fit your needs by adjusting the `log_retention_days` value.
```bash
kubectl apply -f - <<EOF
apiVersion: telemetry.kyma-project.io/v1alpha1
Expand Down
6 changes: 4 additions & 2 deletions docs/user/integration/dynatrace/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ With the Kyma Telemetry module, you gain even more visibility by adding custom s
There are different ways to deploy Dynatrace on Kubernetes. All [deployment options](https://www.dynatrace.com/support/help/setup-and-configuration/setup-on-container-platforms/kubernetes/get-started-with-kubernetes-monitoring/deployment-options-k8s) are based on the [Dynatrace Operator](https://github.com/Dynatrace/dynatrace-operator).
1. Install Dynatrace with the namespace you prepared earlier.
>**NOTE:** By default, Dynatrace uses the classic full-stack injection. However, for better stability, we recommend using the [cloud-native fullstack injection](https://docs.dynatrace.com/docs/setup-and-configuration/setup-on-k8s/installation/cloud-native-fullstack).
> [!NOTE]
> By default, Dynatrace uses the classic full-stack injection. However, for better stability, we recommend using the [cloud-native fullstack injection](https://docs.dynatrace.com/docs/setup-and-configuration/setup-on-k8s/installation/cloud-native-fullstack).
2. In the DynaKube resource, configure the correct `apiurl` of your environment.
Expand Down Expand Up @@ -119,7 +120,8 @@ To start ingesting custom spans and Istio spans, you must enable the Istio traci
The default configuration has the **randomSamplingPercentage** property set to `1.0`, meaning it samples 1% of all requests. To change the sampling rate, adjust the property to the desired value, up to 100 percent.
> **CAUTION:** Be cautious when you configure the **randomSamplingPercentage**:
> [!WARNING]
> Be cautious when you configure the **randomSamplingPercentage**:
> - Could cause high volume of traces.
> - The Kyma trace collector component does not scale automatically.
Expand Down
3 changes: 2 additions & 1 deletion docs/user/integration/jaeger/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ Learn how to use [Jaeger](https://github.com/jaegertracing/helm-charts/tree/main

### Install Jaeger

> **NOTE:** It is officially recommended to install Jaeger with the [Jaeger operator](https://github.com/jaegertracing/helm-charts/tree/main/charts/jaeger-operator). Because the operator requires a cert-manager to be installed, the following instructions use a plain Jaeger installation. However, the described installation is not meant to be used for production setups.
> [!NOTE]
> It is officially recommended to install Jaeger with the [Jaeger operator](https://github.com/jaegertracing/helm-charts/tree/main/charts/jaeger-operator). Because the operator requires a cert-manager to be installed, the following instructions use a plain Jaeger installation. However, the described installation is not meant to be used for production setups.

Run the Helm upgrade command, which installs the chart if not present yet.

Expand Down
9 changes: 6 additions & 3 deletions docs/user/integration/loki/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@

Learn how to use [Loki](https://github.com/grafana/loki/tree/main/production/helm/loki) as a logging backend with Kyma's [LogPipeline](../../02-logs.md) or with [Promtail](https://grafana.com/docs/loki/latest/clients/promtail/).

>**CAUTION:** This guide uses the Grafana Loki version, which is distributed under AGPL-3.0 only and might not be free of charge for commercial usage.
> [!WARNING]
> This guide uses the Grafana Loki version, which is distributed under AGPL-3.0 only and might not be free of charge for commercial usage.
![setup](./../assets/loki.drawio.svg)

Expand Down Expand Up @@ -95,7 +96,8 @@ helm upgrade --install --create-namespace -n ${K8S_NAMESPACE} promtail grafana/p
#### **Install Fluent Bit with Kyma's LogPipeline**

>**CAUTION:** This setup uses an unsupported output plugin for the LogPipeline.
> [!WARNING]
> This setup uses an unsupported output plugin for the LogPipeline.

Apply the LogPipeline:

Expand All @@ -122,7 +124,8 @@ Apply the LogPipeline:
When the status of the applied LogPipeline resource turns into `Running`, the underlying Fluent Bit is reconfigured and log shipment to your Loki instance is active.
>**NOTE:** The used output plugin configuration uses a static label map to assign labels of a Pod to Loki log streams. It's not recommended to activate the `auto_kubernetes_labels` feature for using all labels of a Pod because this lowers the performance. Follow [Loki's labelling best practices](https://grafana.com/docs/loki/latest/best-practices/) for a tailor-made setup that fits your workload configuration.
> [!NOTE]
> The used output plugin configuration uses a static label map to assign labels of a Pod to Loki log streams. It's not recommended to activate the `auto_kubernetes_labels` feature for using all labels of a Pod because this lowers the performance. Follow [Loki's labelling best practices](https://grafana.com/docs/loki/latest/best-practices/) for a tailor-made setup that fits your workload configuration.
<!-- tabs:end -->
Expand Down
12 changes: 9 additions & 3 deletions docs/user/integration/sap-cloud-logging/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,17 @@ SAP Cloud Logging is an instance-based and environment-agnostic observability se
- [Ship Metrics to SAP Cloud Logging](#ship-metrics-to-sap-cloud-logging)
- [Kyma Dashboard Integration](#kyma-dashboard-integration)
- [SAP Cloud Logging Alerts](#sap-cloud-logging-alerts)
- [Import](#import)
- [Recommended Alerts](#recommended-alerts)
- [SAP Cloud Logging Dashboards](#sap-cloud-logging-dashboards)

## Prerequisites

- Kyma as the target deployment environment.
- The [Telemetry module](../../README.md) is [enabled](https://kyma-project.io/#/02-get-started/01-quick-install).
- An instance of SAP Cloud Logging with OpenTelemetry enabled to ingest distributed traces.
>**TIP:** It's recommended to create it with the SAP BTP service operator (see [Create an SAP Cloud Logging Instance through SAP BTP Service Operator](https://help.sap.com/docs/cloud-logging/cloud-logging/create-sap-cloud-logging-instance-through-sap-btp-service-operator?locale=en-US&version=Cloud)), because it takes care of creation and rotation of the required Secret. However, you can choose any other method of creating the instance and the Secret, as long as the parameter for OTLP ingestion is enabled in the instance. For details, see [Configuration Parameters](https://help.sap.com/docs/cloud-logging/cloud-logging/configuration-parameters?locale=en-US&version=Cloud).
> [!TIP]
> It's recommended to create it with the SAP BTP service operator (see [Create an SAP Cloud Logging Instance through SAP BTP Service Operator](https://help.sap.com/docs/cloud-logging/cloud-logging/create-sap-cloud-logging-instance-through-sap-btp-service-operator?locale=en-US&version=Cloud)), because it takes care of creation and rotation of the required Secret. However, you can choose any other method of creating the instance and the Secret, as long as the parameter for OTLP ingestion is enabled in the instance. For details, see [Configuration Parameters](https://help.sap.com/docs/cloud-logging/cloud-logging/configuration-parameters?locale=en-US&version=Cloud).
- A Secret in the respective namespace in the Kyma cluster, holding the credentials and endpoints for the instance. In this guide, the Secret is named `sap-cloud-logging` and the namespace `sap-cloud-logging-integration` as illustrated in this [example](https://github.com/kyma-project/telemetry-manager/blob/main/docs/user/integration/sap-cloud-logging/secret-example.yaml).
<!-- markdown-link-check-disable -->
- Kubernetes CLI (kubectl) (see [Install the Kubernetes Command Line Tool](https://developers.sap.com/tutorials/cp-kyma-download-cli.html)).
Expand Down Expand Up @@ -102,7 +106,8 @@ The Telemetry module supports the convenient shipment of applications and access
1. By default, Istio access logs are disabled in Kyma. To enable Istio access logs selectively for your workload, follow [Enable Istio access logs](https://kyma-project.io/#/istio/user/operation-guides/02-30-enable-istio-access-logs).
As a result, you can analyze access logs in the default SAP Cloud Logging dashboards shipped for SAP BTP, Kyma runtime.
>**CAUTION:** The provided feature uses an Istio API in the alpha state, which may or may not be continued in future releases.
> [!WARNING]
> The provided feature uses an Istio API in the alpha state, which may or may not be continued in future releases.
2. Deploy the LogPipeline for Istio access logs and enable access logs in Kyma:
Expand Down Expand Up @@ -183,7 +188,8 @@ To enable shipping traces to the SAP Cloud Logging service instance, follow this
The default configuration has the **randomSamplingPercentage** property set to `1.0`, meaning it samples 1% of all requests. To change the sampling rate, adjust the property to the desired value, up to 100 percent.
> **CAUTION:** Be cautious when you configure the **randomSamplingPercentage**:
> [!WARNING]
> Be cautious when you configure the **randomSamplingPercentage**:
> - Traces might consume a significant storage volume in Cloud Logging Service.
> - The Kyma trace collector component does not scale automatically.
Expand Down

0 comments on commit d1348ef

Please sign in to comment.