Better checks that static file routing works as expected

[KevinMind]

Relates: mozilla/addons#15066

Description

First add checks to static file handling and better debugging of nginx static file serving

• Removed obsolete file checks and user validation from Makefile-docker.
• Updated Nginx configuration to improve static file handling and added headers for better traceability.
• Refactored storage management commands in the Python codebase:
  • Renamed clean_storage to make_storage for clarity and added a clean parameter.
  • Updated command implementations to use the new make_storage method.
• Introduced a new system check for Nginx configurations to ensure proper file accessibility and response validation.

Second add content hashing of static built directories allowing guarantees that static files created during the build are not modified at runtime.

• Removed the previous build information generation method and replaced it with a new Python script (scripts/build_info.py) that creates a build info file with content hashes for static assets and locales.
• Updated Dockerfile to utilize the new build_info.py script for generating build information during the Docker build process.
• Adjusted the static check in the Django app to validate the content hash of static files against the expected hash.
• Cleaned up .dockerignore and .gitignore files by removing the exclusion of build*.py files.
• Added unit tests for the new build_info.py script and its functionality.

Context

We serve static files from several possible sources. This PR ensures we understand where a file was served from and also have tests ensuring the routing through nginx works as expected.

Testing

Test NGINX routing

CI should suffice as it runs the make checks but you can break the routing in addons.conf and re-run the checks to ensure they fail. Comment out the /static file handler or specific blocks to force nginx to serve no files or files from incorrect sources

Test content hashing

Run in prod mode (make down first to guarantee correct volumes are used)

make down && make up DOCKER_TARGET=production OLYMPIA_DEPS=development OLYMPIA_MOUNT=development

Then shell into the container make shell and run

cat /build-info.json
./scripts/build-info.py --target production

Verify that the content hashes for static-files and locales match in both places. the build-info.json file is from the image build and the script is running against the current state in the container.

You can verify that this breaks the checks by modifying a file in either of the directories

echo "breaks" > /data/olympia/site-static/broken.txt

And then run checks

make check_django

This will raise an error that the content hash has changed which is not allowed on production targets. If you remove the file, the check should then pass again.

Tip

Bonus points

You can freely modify the staticfiles.json file in site-static because this file is non-deterministic and thus excluded from the content hash.

Checklist

• Add #ISSUENUM at the top of your PR to an existing open issue in the mozilla/addons repository.
• Successfully verified the change locally.
• The change is covered by automated tests, or otherwise indicated why doing so is unnecessary/impossible.
• Add before and after screenshots (Only for changes that impact the UI).
• Add or update relevant docs reflecting the changes made.