Add readme with example deployment

README.md

@@ -1,9 +1,7 @@
 # telemetry-controller
-// TODO(user): Add simple overview of use/purpose
-
+The Telemetry Controller is a multi-tenancy focused solution, that facilitates collection of telemetry data from Kubernetes workloads, without any need for changes to the running software.
 ## Description
-// TODO(user): An in-depth paragraph about your project and overview of use
-
+Telemetry-controller can be configured using Custom Resources to set up an opinionated Opentelemetry Collector configuration to route log messages based on rules defined as a Tenant -> Subscription relation map.
 ## Getting Started
 
 ### Prerequisites
@@ -13,13 +11,21 @@
 - Access to a Kubernetes v1.11.3+ cluster.
 
 ### To Deploy on the cluster
+
+**Install cert-manager, and opentelemtry-operator:**
+```sh
+helm upgrade --install --repo https://charts.jetstack.io cert-manager cert-manager --namespace cert-manager --create-namespace --version v1.13.3 --set installCRDs=true --wait
+
+kubectl apply -f https://github.com/open-telemetry/opentelemetry-operator/releases/latest/download/opentelemetry-operator.yaml --wait
+```
+
 **Build and push your image to the location specified by `IMG`:**
 
 ```sh
 make docker-build docker-push IMG=<some-registry>/telemetry-controller:tag
 ```
 
-**NOTE:** This image ought to be published in the personal registry you specified. 
+> **NOTE:** This image ought to be published in the personal registry you specified. 
 And it is required to have access to pull the image from the working environment. 
 Make sure you have the proper permission to the registry if the above commands don’t work.
 
@@ -39,19 +45,42 @@ make deploy IMG=<some-registry>/telemetry-controller:tag
 privileges or be logged in as admin.
 
 **Create instances of your solution**
-You can apply the samples (examples) from the config/sample:
+You can apply the you can deploy the example configuration provided as part of the docs:
 
 ```sh
-kubectl apply -k config/samples/
+# Deploy Openobserve
+kubectl apply -f docs/examples/simple-demo/openobserve.yaml
+
+# Set up portforwarding for Openobserve UI
+kubectl -n openobserve port-forward svc/openobserve 5080:5080 &
 ```
 
->**NOTE**: Ensure that the samples has default values to test it out.
+Open the UI at `localhost:5080`, navigate to the `Ingestion/OTEL Collector` tab, and copy the authorization token as seen on the screenshot.
+![Openobserve auth](docs/assets/openobserve-auth.png)
+
+Paste this token to the example manifests:
+```sh
+sed -i '' -e "s/\<TOKEN\>/INSERT YOUR COPIED TOKEN HERE/" docs/examples/simple-demo/one_tenant_two_subscriptions.yaml
+```
+```sh
+# Deploy the pipeline definition
+kubectl apply -f docs/examples/simple-demo/one_tenant_two_subscriptions.yaml
+```
+This will deploy a demo pipeline with one tenant, two subscriptions, and an OpenObserve instance, where logs are ingested, and visualized.
+
+**Create a workload, which will generate logs for the pipeline:**
+```sh
+helm install --wait --create-namespace --namespace example-tenant-ns --generate-name oci://ghcr.io/kube-logging/helm-charts/log-generator
+```
+
+**Open the Openobserve UI and inspect the generated log messages**
+![Openobserve logs](docs/assets/openobserve-logs.png)
 
 ### To Uninstall
 **Delete the instances (CRs) from the cluster:**
 
 ```sh
-kubectl delete -k config/samples/
+kubectl delete -f docs/simple-demo/
 ```
 
 **Delete the APIs(CRDs) from the cluster:**
@@ -67,11 +96,17 @@ make undeploy
 ```
 
 ## Contributing
-// TODO(user): Add detailed information on how you would like others to contribute to this project
 
-**NOTE:** Run `make --help` for more information on all potential `make` targets
+If you find this project useful, help us:
+
+- Support the development of this project and star this repo! :star:
+- If you use the Logging operator in a production environment, add yourself to the list of production [adopters](https://github.com/kube-logging/logging-operator/blob/master/ADOPTERS.md).:metal: <br> 
+- Help new users with issues they may encounter :muscle:
+- Send a pull request with your new features and bug fixes :rocket: 
+
+Please read the [Organisation's Code of Conduct](https://github.com/kube-logging/.github/blob/main/CODE_OF_CONDUCT.md)!
 
-More information can be found via the [Kubebuilder Documentation](https://book.kubebuilder.io/introduction.html)
+*For more information, read our organization's [contribution guidelines](https://github.com/kube-logging/.github/blob/main/CONTRIBUTING.md)*.
 
 ## License
 

docs/examples/simple-demo/one_tenant_two_subscriptions.yaml

@@ -40,9 +40,9 @@ metadata:
   name: subscription-sample-1
   namespace: example-tenant-ns
 spec:
-  ottl: 'route()'
+  ottl: "route()"
   outputs:
-    - name: otlp-test-output
+    - name: otlp-test-output-1
       namespace: collector
 ---
 apiVersion: telemetry.kube-logging.dev/v1alpha1
@@ -51,19 +51,23 @@ metadata:
   name: subscription-sample-2
   namespace: example-tenant-ns
 spec:
-  ottl: 'route()'
+  ottl: "route()"
   outputs:
     - name: otlp-test-output-2
       namespace: collector
 ---
 apiVersion: telemetry.kube-logging.dev/v1alpha1
 kind: OtelOutput
 metadata:
-  name: otlp-test-output
+  name: otlp-test-output-1
   namespace: collector
 spec:
   otlp:
-    endpoint: receiver-collector.example-tenant-ns.svc.cluster.local:4317
+    endpoint: openobserve-otlp-grpc.openobserve.svc.cluster.local:5081
+    headers:
+      Authorization: "Basic <TOKEN>"
+      organization: default
+      stream-name: default
     tls:
       insecure: true
 ---
@@ -74,6 +78,10 @@ metadata:
   namespace: collector
 spec:
   otlp:
-    endpoint: receiver-collector.example-tenant-ns.svc.cluster.local:4317
+    endpoint: openobserve-otlp-grpc.openobserve.svc.cluster.local:5081
+    headers:
+      Authorization: "Basic <TOKEN>"
+      organization: default
+      stream-name: default
     tls:
       insecure: true

docs/examples/simple-demo/openebserve.yaml

@@ -0,0 +1,93 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: openobserve
+  namespace: openobserve
+spec:
+  clusterIP: None
+  selector:
+    app: openobserve
+  ports:
+    - name: http
+      port: 5080
+      targetPort: 5080
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: openobserve-otlp-grpc
+  namespace: openobserve
+spec:
+  clusterIP: None
+  selector:
+    app: openobserve
+  ports:
+    - name: otlp-grpc
+      port: 5081
+      targetPort: 5081
+---
+# create statefulset
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: openobserve
+  namespace: openobserve
+  labels:
+    name: openobserve
+spec:
+  serviceName: openobserve
+  replicas: 1
+  selector:
+    matchLabels:
+      name: openobserve
+      app: openobserve
+  template:
+    metadata:
+      labels:
+        name: openobserve
+        app: openobserve
+    spec:
+      securityContext:
+        fsGroup: 2000
+        runAsUser: 10000
+        runAsGroup: 3000
+        runAsNonRoot: true
+      # terminationGracePeriodSeconds: 0
+      containers:
+        - name: openobserve
+          image: public.ecr.aws/zinclabs/openobserve:v0.7.2
+          env:
+            - name: ZO_ROOT_USER_EMAIL
+              value: root@example.com
+            - name: ZO_ROOT_USER_PASSWORD
+              value: Complexpass#123
+            - name: ZO_DATA_DIR
+              value: /data
+          # command: ["/bin/bash", "-c", "while true; do sleep 1; done"]
+          imagePullPolicy: Always
+          resources:
+            limits:
+              cpu: 4096m
+              memory: 2048Mi
+            requests:
+              cpu: 256m
+              memory: 50Mi
+          ports:
+            - containerPort: 5080
+              name: http
+            - containerPort: 50801
+              name: otlp-grpc
+          volumeMounts:
+            - name: data
+              mountPath: /data
+  volumeClaimTemplates:
+    - metadata:
+        name: data
+      spec:
+        accessModes:
+          - ReadWriteOnce
+        # storageClassName: default
+        # NOTE: You can increase the storage size
+        resources:
+          requests:
+            storage: 10Gi

e2e/e2e_test.sh

@@ -38,7 +38,7 @@ kubectl wait --namespace opentelemetry-operator-system --for=condition=available
 (cd .. && make manifests generate install)
 
 # Use example
-kubectl apply -f ../docs/examples/simple-demo
+kubectl apply -f ../e2e/testdata/one_tenant_two_subscriptions
 
 if [[ -z "${CI_MODE}" ]]; then
   $(cd .. && timeout 5m make run &)

e2e/testdata/one_tenant_two_subscriptions/one_tenant_two_subscriptions.yaml

@@ -0,0 +1,79 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: collector
+---
+apiVersion: v1
+kind: Namespace
+metadata:
+  labels:
+    nsSelector: example-tenant
+  name: example-tenant-ns
+---
+apiVersion: telemetry.kube-logging.dev/v1alpha1
+kind: Collector
+metadata:
+  name: example-collector
+spec:
+  controlNamespace: collector
+  tenantSelector:
+    matchLabels:
+      collectorLabel: example-collector
+---
+apiVersion: telemetry.kube-logging.dev/v1alpha1
+kind: Tenant
+metadata:
+  labels:
+    collectorLabel: example-collector
+  name: example-tenant
+spec:
+  subscriptionNamespaceSelectors:
+    - matchLabels:
+        nsSelector: example-tenant
+  logSourceNamespaceSelectors:
+    - matchLabels:
+        nsSelector: example-tenant
+---
+apiVersion: telemetry.kube-logging.dev/v1alpha1
+kind: Subscription
+metadata:
+  name: subscription-sample-1
+  namespace: example-tenant-ns
+spec:
+  ottl: 'route()'
+  outputs:
+    - name: otlp-test-output
+      namespace: collector
+---
+apiVersion: telemetry.kube-logging.dev/v1alpha1
+kind: Subscription
+metadata:
+  name: subscription-sample-2
+  namespace: example-tenant-ns
+spec:
+  ottl: 'route()'
+  outputs:
+    - name: otlp-test-output-2
+      namespace: collector
+---
+apiVersion: telemetry.kube-logging.dev/v1alpha1
+kind: OtelOutput
+metadata:
+  name: otlp-test-output
+  namespace: collector
+spec:
+  otlp:
+    endpoint: receiver-collector.example-tenant-ns.svc.cluster.local:4317
+    tls:
+      insecure: true
+---
+apiVersion: telemetry.kube-logging.dev/v1alpha1
+kind: OtelOutput
+metadata:
+  name: otlp-test-output-2
+  namespace: collector
+spec:
+  otlp:
+    endpoint: receiver-collector.example-tenant-ns.svc.cluster.local:4317
+    tls:
+      insecure: true