HDDS-11631. Add Run with Docker Compose guide #101

ptlrs · 2024-10-31T05:46:54Z

What changes were proposed in this pull request?

Added Run with Docker Compose guide to the Developer Guide section

Please describe your PR in detail:

Added a Run with Docker compose guide
Added new keywords to fix spelling linter errors
Added a placeholder page to document docker/docker-compose related issues
Added mermaid package to render images via pnpm add which updated package.json and pnpm-lock.yaml
Updated the pnpm version to fix incompatible lock file version error when running docker-compose

What is the link to the Apache Jira?

https://issues.apache.org/jira/browse/HDDS-11631

How was this patch tested?

Ran the website locally via docker-compose
CI:
~~https://github.com/ptlrs/ozone-site/actions/runs/11606604449~~
~~https://github.com/ptlrs/ozone-site/actions/runs/11845847017~~
~~https://github.com/ptlrs/ozone-site/actions/runs/12627156091~~
https://github.com/ptlrs/ozone-site/actions/runs/12627665227

Ran the website locally:

errose28

Thanks for adding this @ptlrs. There's some more information we can add to this page too:

Provide a quick summary of the various compose clusters available for HA and security.
Explain how to make configuration changes with the docker-config file.

cspell.yaml

docs/06-troubleshooting/15-docker.md

docs/08-developer-guide/02-run/02-docker-compose.md

errose28 · 2024-11-06T00:17:41Z

package.json

    "@docusaurus/plugin-pwa": "3.3.2",
+    "@docusaurus/preset-classic": "3.3.2",
+    "@docusaurus/theme-mermaid": "3.3.2",


I checked out this change and ran docker compse up --build to build an image with the new dependencies but got this error. Not sure exactly what the cause is:

ozone-site-site-1 | ozone-site-site-1 | > doc-site@0.1.0 start /ozone-site ozone-site-site-1 | > docusaurus start --port 3001 "--host" "0.0.0.0" ozone-site-site-1 | ozone-site-site-1 | [INFO] Starting the development server... ozone-site-site-1 | ozone-site-site-1 | [ERROR] Error: Docusaurus was unable to resolve the "@docusaurus/theme-mermaid" theme. Make sure one of the following packages are installed: ozone-site-site-1 | - @docusaurus/theme-mermaid ozone-site-site-1 | - @docusaurus/docusaurus-theme-theme-mermaid ozone-site-site-1 | at resolveModuleName (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/moduleShorthand.js:41:15) ozone-site-site-1 | at normalizeShorthand (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/configs.js:76:60) ozone-site-site-1 | at /ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/configs.js:90:63 ozone-site-site-1 | at Array.map (<anonymous>) ozone-site-site-1 | at loadPluginConfigs (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/configs.js:90:48) ozone-site-site-1 | at async initPlugins (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/init.js:42:27) ozone-site-site-1 | at async /ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/plugins/plugins.js:136:36 ozone-site-site-1 | at async loadSite (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/server/site.js:127:45) ozone-site-site-1 | at async createReloadableSite (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/commands/start/utils.js:46:16) ozone-site-site-1 | at async Command.start (/ozone-site/node_modules/.pnpm/@docusaurus+core@3.3.2_@docusaurus+types@3.3.2_react-dom@18.2.0_react@18.2.0_typescript@5.4.5/node_modules/@docusaurus/core/lib/commands/start/start.js:22:28) ozone-site-site-1 | [INFO] Docusaurus version: 3.3.2 ozone-site-site-1 | Node version: v20.18.0 ozone-site-site-1 | ELIFECYCLE Command failed with exit code 1. ozone-site-site-1 exited with code 1

I saw that I faced the same problem after uninstalling docusaurus/theme-mermaid.
I was able to resolve it by reinstalling the package again on my mac.
Next, I ran docker compse up --build and then ran it with no issues.

errose28 · 2024-11-06T00:18:07Z

package.json

@@ -1,5 +1,5 @@
 {
-  "packageManager": "pnpm@8.7.1",
+  "packageManager": "pnpm@9.12.3",


Can you clarify why we need the pnpm version bump? Is mermaid incompatible with older versions?

With pnpm@8.7.1, running docker-compose fails with the following error:

0.280 ! Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-8.7.1.tgz 1.109 ERR_PNPM_LOCKFILE_BREAKING_CHANGE Lockfile /ozone-site/pnpm-lock.yaml not compatible with current pnpm 1.109 1.109 Run with the --force parameter to recreate the lockfile. ------ failed to solve: process "/bin/sh -c pnpm install --prod --frozen-lockfile" did not complete successfully: exit code: 1 `docker-compose` process finished with exit code 17

I added the docusaurus/theme-mermaid package via pnpm add @docusaurus/theme-mermaid@3.3.2 on my mac. This updated the pnpm-lock.yaml and the package.json files.

errose28 · 2024-11-06T00:30:30Z

docs/08-developer-guide/02-run/02-docker-compose.md

+
+This will pull the official Apache Ozone images from Docker Hub.
+
+## Architecture Diagram


To avoid duplicate content I think we should leave this information for the page dedicated to Ozone architecture. Default ports will be covered in docs/administrator-guide/configuration/basic/network/. This may mean we don't need to add mermaid in this PR.

As a new user, it is not immediately apparent how the Ozone services will be divvied across containers. It could be the case that all Ozone services run within one container. There are several possible configurations.

I added the mermaid diagram to visualize how many containers will be available and how each Ozone service maps to those containers.

It just so happens that each Ozone service is a separate container and the container diagram looks similar to what could easily have been an Ozone architecture diagram. Perhaps the title and wording add to the confusion.

If it is still a cause of confusion, I can remove the diagram. Not a problem.

docs/08-developer-guide/02-run/02-docker-compose.md

adoroszlai

I suggest writing the page on Docker for users first. We should refer developers to that guide for basics.

errose28 · 2024-11-14T22:52:17Z

Good suggestion Attila. Rishabh and I talked on a call and he had the idea of creating a new compose file with all the configs baked so that new users can pull just that one file instead of a whole release tarball where the configs live outside the docker compose definition. For now when writing the docker quick start we can refer to the current process of pulling a full release though.

ptlrs · 2025-01-06T05:30:03Z

I suggest writing the page on Docker for users first. We should refer developers to that guide for basics.

The PR #122 adds the Try Ozone With Docker page

…se-guide' into HDDS-11631-run-with-docker-compose-guide

ptlrs added 2 commits October 30, 2024 21:58

HDDS-11631.Added Running Ozone From Docker Compose guide

c31c52f

HDDS-11631. Fix linter errors

8554c03

github-actions bot added the website-v2 Tasks for the new ozone website targeting the HDDS-9225-website-v2 branch label Oct 31, 2024

errose28 self-requested a review November 1, 2024 17:31

errose28 added the docs Changes updating documentation on the website label Nov 1, 2024

errose28 reviewed Nov 6, 2024

View reviewed changes

ptlrs added 2 commits November 12, 2024 23:48

HDDS-11631. Fix review comments

8d51361

HDDS-11631. Fix review comments

3ac2f6d

ptlrs requested a review from errose28 November 14, 2024 21:32

adoroszlai reviewed Nov 14, 2024

View reviewed changes

HDDS-12024. Link to the Try Ozone with Docker page

a6c94fc

ptlrs requested a review from adoroszlai January 6, 2025 05:35

ptlrs added 3 commits January 5, 2025 22:22

HDDS-11631. Link to the Try Ozone with Docker page

7461fe7

HDDS-11631. Fix review comments

ac2374c

Merge remote-tracking branch 'origin/HDDS-11631-run-with-docker-compo…

787bfa6

…se-guide' into HDDS-11631-run-with-docker-compose-guide

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDDS-11631. Add Run with Docker Compose guide #101

HDDS-11631. Add Run with Docker Compose guide #101

ptlrs commented Oct 31, 2024 •

edited

Loading

errose28 left a comment

errose28 Nov 6, 2024

ptlrs Nov 14, 2024

errose28 Nov 6, 2024

ptlrs Nov 13, 2024

errose28 Nov 6, 2024

ptlrs Nov 13, 2024

adoroszlai left a comment

errose28 commented Nov 14, 2024

ptlrs commented Jan 6, 2025


		This will pull the official Apache Ozone images from Docker Hub.

		## Architecture Diagram

HDDS-11631. Add Run with Docker Compose guide #101

Are you sure you want to change the base?

HDDS-11631. Add Run with Docker Compose guide #101

Conversation

ptlrs commented Oct 31, 2024 • edited Loading

What changes were proposed in this pull request?

What is the link to the Apache Jira?

How was this patch tested?

errose28 left a comment

Choose a reason for hiding this comment

errose28 Nov 6, 2024

Choose a reason for hiding this comment

ptlrs Nov 14, 2024

Choose a reason for hiding this comment

errose28 Nov 6, 2024

Choose a reason for hiding this comment

ptlrs Nov 13, 2024

Choose a reason for hiding this comment

errose28 Nov 6, 2024

Choose a reason for hiding this comment

ptlrs Nov 13, 2024

Choose a reason for hiding this comment

adoroszlai left a comment

Choose a reason for hiding this comment

errose28 commented Nov 14, 2024

ptlrs commented Jan 6, 2025

ptlrs commented Oct 31, 2024 •

edited

Loading