Optimize docker image size

[jantuomi]

Problem

The current Dockerfile works well but has some issues:

• It uses a large base image, node:21-slim
• It copies the whole workspace context to the image, which increases the size as well as the chance of accidentally baking in secrets, like env files (COPY ./ ./)
• It does not use multi-stage builds and retains all dev dependencies during runtime
• It has NextJS telemetry enabled
• It requires specific --build-args to be supplied but does not actually do anything with them
• It installs prisma manually since it is not a runtime dependency in package.json

A built image on linux/amd64 is around 1.2GB in size. This size can be reduced significantly.

Changes

This PR addresses each of the issues and reduces the image size from 1.2GB to a nice 200MB. Changes include:

• Use node:21-alpine for a smaller base image
• Copy workspace objects into the image specifying each object separately
• Use multi-stage builds to ensure that only runtime dependencies and the minimal set of application files are included in the resulting deployable image
• Disable NextJS telemetry
• Set placeholder POSTGRES_* env vars in the Dockerfile itself so that the user does not have to remember to supply them
• Move prisma from dev dependencies to actual dependencies in package.json

The multi-stage build is set up like so:

• base stage
  • Copy workspace application files into the image
  • Install dev dependencies
  • Invoke npm run build
• runtime-deps stage
  • Copy the built NextJS application from base
  • Install only the runtime dependencies
  • Compress the application into a zstd archive, reducing its file size significantly
• runner stage
  • Copy the compressed archive from runtime-deps
  • Copy the entrypoint scripts from the workspace
  • Upon container startup, decompress the archive and start the application

The compression/decompression steps contribute an around 200MB size reduction, using the compression algorithm zstd, which is a great balance of size reduction and processing speed. However, they cause a couple seconds of processing time both at compression time (during build) and at decompression time (during first container startup). This is time that is saved in disk and network I/O.

Whether or not to include the compression/decompression may be controversial and can be removed if it is not seen as a good fit. I personally think it is a good thing to include.

[justcallmelarry]

This file does not exist as far as I can tell? So my test build of your branch is failing
Maybe I'm missing something

[justcallmelarry]

I tried creating an empty file with the name just to get further, and it seems like it's gitignored, so I guess this is some sort of local file that I don't have (and which would not be present should a GHA be introduced to build images automatically or similar).

[jantuomi]

Good catch, that seems like a generated file. I'll remove that line 👍

[justcallmelarry]

I like most of the changes (except that I'm not all that sold on the compression parts, but this feels very subjective)!
I have #73 open to try to allow the next public (at least) env vars to be present at build time, since they are not respected if they are not, and the solution introduced here would just add more and more env vars as they are introduced to the Dockerfile, which seems a bit messy.

If my changes or something similar could be incorporated in this PR, that PR could be closed down in favour of these changes.

You probably have a better understanding of what the best practices around those things than I do, so if there is another solution that brings about the same result I would be equally good with that.

Currently have tried building this branch locally with a completely empty next-env.d.ts (i just ran touch) and if I add NEXT_PUBLIC_ENABLE_EXPENSE_DOCUMENTS to the container (along with the needed S3_UPLOAD-vars, no expense attachment option is shown.
This is exactly the same functionality as currently, so I'm not saying this needs to be fixed specifically in this PR
My main concern is just adding a long list of mocked env vars right into the Dockerfile

[jantuomi]

I like most of the changes (except that I'm not all that sold on the compression parts, but this feels very subjective)! I have #73 open to try to allow the next public (at least) env vars to be present at build time, since they are not respected if they are not, and the solution introduced here would just add more and more env vars as they are introduced to the Dockerfile, which seems a bit messy.

If my changes or something similar could be incorporated in this PR, that PR could be closed down in favour of these changes.

You probably have a better understanding of what the best practices around those things than I do, so if there is another solution that brings about the same result I would be equally good with that.

Currently have tried building this branch locally with a completely empty next-env.d.ts (i just ran touch) and if I add NEXT_PUBLIC_ENABLE_EXPENSE_DOCUMENTS to the container (along with the needed S3_UPLOAD-vars, no expense attachment option is shown. This is exactly the same functionality as currently, so I'm not saying this needs to be fixed specifically in this PR My main concern is just adding a long list of mocked env vars right into the Dockerfile

Thanks for the review. I think I'll drop the compression stuff for now, I can later open a new PR for them specifically so that the other changes are not blocked. I'll take a look at #73 and check if there's an elegant solution that could be implemented on top of this PR.

[jantuomi]

@justcallmelarry I made some changes:

• Remove compression steps. The image is now around 400MB.
• Add contents of Add an env file with mocked env vars added for Docker production builds #73
• Use server actions to get env feature flags during runtime

With these changes, the three environment variables NEXT_PUBLIC_ENABLE_EXPENSE_DOCUMENTS, NEXT_PUBLIC_ENABLE_CATEGORY_EXTRACT and NEXT_PUBLIC_ENABLE_RECEIPT_EXTRACT are properly evaluated during runtime, and the user can configure those settings at container startup time. This is nice, since now changing those settings does not mandate a rebuild.

The fix works by directly reading the global.process.env namespace in a server action. The regular process.env has the build-time baked values but this one has the runtime ones. Kind of a weird quirk of NextJS if you ask me.

Note: the zod validations for these environment variables work, but seem to only trigger when accessing the app, not when starting up the container.

[justcallmelarry]

@jantuomi nice 👍

I am sure that just having the PUBLIC_* env var set during build works, since that's how I'm running my own instance (set to false at build, then changed via container.env at runtime), and I'm not really great at next/ts in general, so I won't comment on that fix overall.

I do think that the docker changes are great, and was trying to reach a similar result myself, but the aforementioned lack of knowledge of next made it a huge trial-and-error-process that did not bear fruit.

if Sebastien thinks the ts-parts looks good, I can at least vouch for the docker-related parts

[jantuomi]

I found a part of the reason for our confusion: in the expression {process.env.NEXT_PUBLIC_ENABLE_EXPENSE_DOCUMENTS && <Card ... />}, the env var is a string. In JS, only the empty string is falsy. As such, the string "false" will be evaluated as true. I think there is something related to this happening in your environment.

Can you try manually specifying NEXT_PUBLIC_ENABLE_EXPENSE_DOCUMENTS=false in container.env and see if the expense document component is rendered?

A common pattern regarding env vars is to parse the strings to a proper data object, similar to what is going on in the env schema with Zod. Then, in application code, you would only refer to this parsed object and not the raw env vars from process.env.

[justcallmelarry]

I found a part of the reason for our confusion: in the expression {process.env.NEXT_PUBLIC_ENABLE_EXPENSE_DOCUMENTS && <Card ... />}, the env var is a string. In JS, only the empty string is falsy. As such, the string "false" will be evaluated as true. I think there is something related to this happening in your environment.

Can you try manually specifying NEXT_PUBLIC_ENABLE_EXPENSE_DOCUMENTS=false in container.env and see if the expense document component is rendered?

A common pattern regarding env vars is to parse the strings to a proper data object, similar to what is going on in the env schema with Zod. Then, in application code, you would only refer to this parsed object and not the raw env vars from process.env.

ah, that makes sense, i have NEXT_PUBLIC_ENABLE_CATEGORY_EXTRACT set to false, but I still get the spinner for category when i enter a title, so this also explains that behaviour

[jantuomi]

The string/bool coercion issue is present in the env Zod validations as well. I added some preprocessing to those env vars so that they are validated correctly.

I used the parsed Zod object to provide the runtime env. We have to wrap the reference to env in a server action (snippet) to force NextJS to evaluate the env on the server. I don't think there's a way around that. Other than that, this is pretty elegant imo.

import { env } from '@/lib/env'
async function getEnvOnServer() {
    'use server'
    return env;
}

With these changes, the env vars seem to work.

[justcallmelarry]

The string/bool coercion issue is present in the env Zod validations as well. I added some preprocessing to those env vars so that they are validated correctly.

I used the parsed Zod object to provide the runtime env. We have to wrap the reference to env in a server action (snippet) to force NextJS to evaluate the env on the server. I don't think there's a way around that. Other than that, this is pretty elegant imo.

import { env } from '@/lib/env'
async function getEnvOnServer() {
    'use server'
    return env;
}

With these changes, the env vars seem to work.

hmm, I might misunderstand how this works, but now it kinda looks like all of the env vars are exposed to the frontend code?
if this makes the env object available in browser then some of the secrets are pretty bad to expose

[jantuomi]

I think you understand correctly 😅 we obviously need to only allow the NEXT_PUBLIC_* vars through. Allowing everything would be pretty bad

[justcallmelarry]

I think you understand correctly 😅 we obviously need to only allow the NEXT_PUBLIC_* vars through. Allowing everything would be pretty bad

That's why I thought the previous solution looked nice from my perspective 😄

[jantuomi]

Alright, I brought back the feature flags object, this time using the Zod object behind the scenes 👍

[scastiel]

Amazing! Thanks @jantuomi and @justcallmelarry for this ❤️

You’ll just need to format the changes with Prettier (npx prettier -w src) to make the pipeline pass :)

[justcallmelarry]

amaze! 💯

[jantuomi]

With the latest changes I think this is ready for merging 👍