docs: Added Simple Scalable Deployment guide for AWS #14327

Jayclifford345 · 2024-10-01T09:41:32Z

This PR adds a guide to deploying the official Loki helm to AWS in Simple Scalable. It is worth noting that there are lot of AWS specific commands within this guide. This is hard to escape due to the nature of how Loki is deployed to AWS. I have added cavitates throughout the guide to help reduce the support burden on AWS-specific questions.

I am open to removing these sections but feel the guide would be insufficient for a user attempting to deploy Loki on AWS.

Checklist

Reviewed the CONTRIBUTING.md guide (required)
Documentation added
Tests updated
Title matches the required conventional commits format, see here
- Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

docs/sources/setup/install/helm/install-scalable/_index.md

docs/sources/setup/install/helm/install-scalable/aws.md

tomglenn

Awesome work @Jayclifford345 ! 🎉
Left some comments and suggestions. 😄

docs/sources/setup/install/helm/install-microservices/_index.md

docs/sources/setup/install/helm/install-scalable/aws.md

Co-authored-by: Tom Glenn <289945+tomglenn@users.noreply.github.com>

trevorwhitney

Looks great Jay! Just a few small comments.

docs/sources/setup/install/helm/install-microservices/_index.md

docs/sources/setup/install/helm/install-scalable/_index.md

docs/sources/setup/install/helm/install-scalable/aws.md

trevorwhitney · 2024-10-07T22:47:04Z

docs/sources/setup/install/helm/install-scalable/aws.md

+```bash
+aws s3api create-bucket --bucket loki-aws-bucket --region eu-west-2 --create-bucket-configuration LocationConstraint=eu-west-2
+```
+Make sure to replace the region and bucket name with your desired values. We will revisit the bucket policy later in this guide.


and reiterating, making sure the values used in this doc are claimed by Grafana Labs

docs/sources/setup/install/helm/install-scalable/aws.md

poyzannur

great work Jay, thanks a lot!
Minor comments to clarify examples.

docs/sources/setup/install/helm/install-microservices/_index.md

docs/sources/setup/install/helm/install-scalable/aws.md

poyzannur · 2024-10-07T15:26:57Z

docs/sources/setup/install/helm/install-scalable/aws.md

+```bash
+aws s3api create-bucket --bucket loki-aws-bucket --region eu-west-2 --create-bucket-configuration LocationConstraint=eu-west-2
+```
+Make sure to replace the region and bucket name with your desired values. We will revisit the bucket policy later in this guide.


+1 I added a suggestion above.

docs/sources/setup/install/helm/install-microservices/_index.md

docs/sources/setup/install/helm/install-scalable/aws.md

Co-authored-by: Poyzan <31743851+poyzannur@users.noreply.github.com>

Jayclifford345 · 2024-10-08T12:47:59Z

Hey @tomglenn, @poyzannur, @trevorwhitney thank you ever so much for the technical reviews. That should be all of your suggestions added. I shall let @JStickler give it the full docs review next and see where we are at. Then I can start filming the video

moxious

great PR. Some minor things found but overall a massive improvement/contribution to what's there

moxious · 2024-10-08T18:11:48Z

docs/sources/setup/install/helm/_index.md

@@ -22,6 +22,17 @@ This guide references the Loki Helm chart version 3.0 or greater and contains th

 If you are installing Grafana Enterprise Logs, follow the [GEL Helm installation](https://grafana.com/docs/enterprise-logs/<ENTERPRISE_LOGS_VERSION>/setup/helm/).

+## Recommended Installation
+
+The recommended installation method for initial deployments is to use the [Loki Simple Scalable Helm chart]({{< relref "./install-scalable" >}}). This chart provides a simple scalable deployment mode for Loki, separating execution paths into read, write, and backend targets. For small to medium-sized deployments, this chart is a good starting point.


great specific opinionated starting point. I'm not wondering what you think I should do.

moxious · 2024-10-08T18:30:23Z

docs/sources/setup/install/helm/install-microservices/_index.md

@@ -21,16 +21,17 @@ The default Helm chart deploys the following components:
 - **QueryFrontend component** (2 replicas, maxUnavailable: 1): Manages frontend queries. Up to 1 replica can be unavailable during updates.
 - **QueryScheduler component** (2 replicas): Schedules queries.

-It is not recommended to run scalable mode with `filesystem` storage. For the purpose of this guide, we will use MinIO as the object storage to provide a complete example. 
+{{< admonition type="note" >}}


this doesn't necessarily require a change, but I'm curious why this is the right spot to make this point.

I understand it's important and why it's here, but an alternative could be to guide me down the path of the opinionated right way of doing things, and simply provide a summary block somewhere else saying "we chose to embed the following best practices in this approach"

moxious · 2024-10-08T18:33:20Z

docs/sources/setup/install/helm/install-microservices/_index.md

@@ -165,7 +170,7 @@ It is not recommended to run scalable mode with `filesystem` storage. For the pu

 ## Object Storage Configuration

-After testing Loki with MinIO, it is recommended to configure Loki with an object storage provider. The following examples shows how to configure Loki with different object storage providers:
+After testing Loki with MinIO, we recommend to configure Loki with an object storage provider. The following examples shows how to configure Loki with different object storage providers:


suggest link on word MinIO - https://min.io/docs/minio/kubernetes/upstream/index.html

very minor point but it's more than possible people won't know what this is.

moxious · 2024-10-08T18:34:08Z

docs/sources/setup/install/helm/install-microservices/_index.md

@@ -165,7 +170,7 @@ It is not recommended to run scalable mode with `filesystem` storage. For the pu

 ## Object Storage Configuration

-After testing Loki with MinIO, it is recommended to configure Loki with an object storage provider. The following examples shows how to configure Loki with different object storage providers:
+After testing Loki with MinIO, we recommend to configure Loki with an object storage provider. The following examples shows how to configure Loki with different object storage providers:

 {{< admonition type="caution" >}}
 When deploying Loki using S3 Storage **DO NOT** use the default bucket names;  `chunk`, `ruler` and `admin`. Choose a unique name for each bucket. For more information see the following [security update](https://grafana.com/blog/2024/06/27/grafana-security-update-grafana-loki-and-unintended-data-write-attempts-to-amazon-s3-buckets/). This caution does not apply when you are using MinIO. When using MinIO we recommend using the default bucket names.


outside the scope of this PR so ignore me but we probably shouldn't ever have this in the docs. The helm chart should ideally block you from foot-gunning in this particular way

moxious · 2024-10-08T18:34:51Z

docs/sources/setup/install/helm/install-microservices/_index.md

+          period: 24h
+  storage_config:
+    aws:
+      region: <AWS region your bucket is in eg. `eu-west-2`>


Change to

region: eu-west-2. # Change to suit your AWS region

for parallel structure with next line

moxious · 2024-10-08T18:35:44Z

docs/sources/setup/install/helm/install-scalable/_index.md


-It is not recommended to run scalable mode with `filesystem` storage. For the purpose of this guide, we will use MinIO as the object storage to provide a complete example. 
+{{< admonition type="note" >}}


see prevous

moxious · 2024-10-08T18:37:30Z

docs/sources/setup/install/helm/install-scalable/_index.md

+We are activley working on providing more guides for deploying Loki in production. 
+{{< /admonition >}}
+
+It is recommended to run Loki at scale within in a cloud enviroment like AWS, Azure, or GCP. The below guides will show you how to deploy a minimally viable production environment.


Passive language. "We recommend"

https://grafana.com/docs/writers-toolkit/write/style-guide/style-conventions/#write-in-active-voice

moxious · 2024-10-08T18:38:10Z

docs/sources/setup/install/helm/install-scalable/aws.md

+The Loki Gateway service is a LoadBalancer service that exposes the Loki gateway to the internet. This is where you will write logs to and query logs from. By default NGINX is used as the gateway.
+
+{{< admonition type="caution" >}}
+The Loki Gateway service is exposed to the internet. It is recommended to secure the gateway with authentication. Refer to the [Authentication]({{< relref "../../../../operations/authentication" >}}) documentation for more information.


passive voice. "We recommend" to secure.

https://grafana.com/docs/writers-toolkit/write/style-guide/style-conventions/#write-in-active-voice

moxious · 2024-10-08T18:41:01Z

docs/sources/setup/install/helm/install-scalable/aws.md

+
+```
+
+{{< admonition type="caution" >}}


haha but good. I've never blindly copypastad something off of a website before. Why are you looking at me like that?

added SSD guide for AWS

d8dd1c7

Jayclifford345 requested a review from a team as a code owner October 1, 2024 09:41

pull-request-size bot added the size/XL label Oct 1, 2024

Merge branch 'main' into aws-helm

fccc7dc

github-actions bot added the type/docs Issues related to technical documentation; the Docs Squad uses this label across many repositories label Oct 1, 2024

github-actions bot reviewed Oct 1, 2024

View reviewed changes

Jayclifford345 requested a review from JStickler October 1, 2024 09:43

Jayclifford345 added 4 commits October 1, 2024 10:46

fixed title

3305c96

fixed trailing slashes

ff2b1db

updated microservice ref

933ee34

updated index

edf4605

tomglenn reviewed Oct 1, 2024

View reviewed changes

Jayclifford345 and others added 12 commits October 1, 2024 13:26

Update docs/sources/setup/install/helm/install-microservices/_index.md

7886f0d

Co-authored-by: Tom Glenn <289945+tomglenn@users.noreply.github.com>

Update docs/sources/setup/install/helm/install-scalable/aws.md

c7cd341

Co-authored-by: Tom Glenn <289945+tomglenn@users.noreply.github.com>

Update docs/sources/setup/install/helm/install-scalable/aws.md

bcbc2af

Co-authored-by: Tom Glenn <289945+tomglenn@users.noreply.github.com>

Update docs/sources/setup/install/helm/install-scalable/aws.md

a94257c

Co-authored-by: Tom Glenn <289945+tomglenn@users.noreply.github.com>

Update docs/sources/setup/install/helm/install-scalable/aws.md

119676b

Co-authored-by: Tom Glenn <289945+tomglenn@users.noreply.github.com>

Merge branch 'main' into aws-helm

af08bcc

Update docs/sources/setup/install/helm/install-scalable/aws.md

108a3f0

Co-authored-by: Tom Glenn <289945+tomglenn@users.noreply.github.com>

added recommended changes

f77d2d7

Merge branch 'aws-helm' of https://github.com/grafana/loki into aws-helm

6b72e1e

fixed formatting

7103f1c

fixed namespace

8225c4c

added suggestion around seperating chunk and ruler bucket

271ef6d

trevorwhitney approved these changes Oct 7, 2024

View reviewed changes

Jayclifford345 and others added 2 commits October 8, 2024 10:49

Added suggestions by trevor

d9767d2

Merge branch 'main' into aws-helm

ae277ce

poyzannur approved these changes Oct 8, 2024

View reviewed changes

Update docs/sources/setup/install/helm/install-microservices/_index.md

f553894

Co-authored-by: Poyzan <31743851+poyzannur@users.noreply.github.com>

added Poyzans recommendations

dd57de1

moxious reviewed Oct 8, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: Added Simple Scalable Deployment guide for AWS #14327

docs: Added Simple Scalable Deployment guide for AWS #14327

Jayclifford345 commented Oct 1, 2024

tomglenn left a comment

trevorwhitney left a comment

trevorwhitney Oct 7, 2024

poyzannur left a comment

poyzannur Oct 7, 2024

Jayclifford345 commented Oct 8, 2024

moxious left a comment

moxious Oct 8, 2024

moxious Oct 8, 2024

moxious Oct 8, 2024

moxious Oct 8, 2024

moxious Oct 8, 2024

moxious Oct 8, 2024

moxious Oct 8, 2024

moxious Oct 8, 2024

moxious Oct 8, 2024


		It is not recommended to run scalable mode with `filesystem` storage. For the purpose of this guide, we will use MinIO as the object storage to provide a complete example.
		{{< admonition type="note" >}}

docs: Added Simple Scalable Deployment guide for AWS #14327

Are you sure you want to change the base?

docs: Added Simple Scalable Deployment guide for AWS #14327

Conversation

Jayclifford345 commented Oct 1, 2024

tomglenn left a comment

Choose a reason for hiding this comment

trevorwhitney left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

poyzannur left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Jayclifford345 commented Oct 8, 2024

moxious left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment