Merge branch 'main' of https://github.com/mmguero-dev/Malcolm into st…

…aging

config/env-var-actions.yml

@@ -0,0 +1,49 @@
+copied_environment_variables:
+  # destination_env_file:
+  #   source_env_file:
+  #     destination_key:
+  #       source_key
+  netbox_postgres:
+    netbox:
+      DB_HOST:
+        DB_HOST
+    netbox_postgres:
+      DB_NAME:
+        POSTGRES_DB
+      DB_USER:
+        POSTGRES_USER
+      DB_PASSWORD:
+        POSTGRES_PASSWORD
+  redis:
+    netbox:
+      REDIS_HOST:
+        REDIS_HOST
+      REDIS_CACHE_HOST:
+        REDIS_CACHE_HOST
+    netbox_secret:
+      REDIS_PASSWORD:
+        REDIS_PASSWORD
+removed_environment_variables:
+  # env_file:
+  #   - key1
+  #   - key2
+  lookup_common:
+    - CONNECTION_SECONDS_SEVERITY_THRESHOLD
+  netbox_redis:
+    - REDIS_PASSWORD
+  netbox_common:
+    - NETBOX_REDIS_DISABLED
+  netbox_secret:
+    - DB_PASSWORD
+    - DB_USER
+    - REDIS_CACHE_PASSWORD
+    - REDIS_PASSWORD
+  netbox:
+    - DB_HOST
+    - DB_NAME
+    - REDIS_CACHE_HOST
+    - REDIS_HOST
+  netbox_redis_cache:
+    - REDIS_HOST
+    - REDIS_CACHE_HOST
+    - REDIS_PASSWORD

config/redis.env.example

@@ -1,4 +1,6 @@
 REDIS_HOST=redis
+REDIS_PORT=6379
 REDIS_CACHE_HOST=redis-cache
+REDIS_CACHE_PORT=6379
 REDIS_PASSWORD=XXXXXXXXXXXXXXXXXXXXXXXX
 K8S_SECRET=True

docs/README.md

@@ -137,7 +137,7 @@ Malcolm can also easily be deployed locally on an ordinary consumer workstation
 * [Deploying Malcolm in Other Third-Party Environments](third-party-envs.md#ThirdPartyEnv)
 * [Hardening](hardening.md#Hardening)
     - [Compliance Exceptions](hardening.md#ComplianceExceptions)
-* [Installation example using Ubuntu 22.04 LTS](ubuntu-install-example.md#InstallationExample)
+* [Installation example using Ubuntu 24.04 LTS](ubuntu-install-example.md#InstallationExample)
 * [Upgrading Malcolm](malcolm-upgrade.md#UpgradePlan)
 * [Modifying or Contributing to Malcolm](contributing-guide.md#Contributing)
     - [Local modifications](contributing-local-modifications.md#LocalMods)

docs/contributing-guide.md

@@ -28,4 +28,5 @@ It is recommended before reviewing this guide to read the documentation on [cust
 * [Style](contributing-style.md#Style)
 * [Using GitHub runners to build Malcolm images](contributing-github-runners.md#GitHubRunners)
 * [Preparing a Malcolm Release](contributing-release-prep.md)
+* [Malcolm code provenance and software supply chain](contributing-guide-code-provenance.md)
 * [`malcolm-test`: Malcolm System Tests](contributing-malcolm-test.md#MalcolmTest)

docs/custom-rules.md

@@ -51,12 +51,24 @@ On [Hedgehog Linux](hedgehog.md), the Arkime Lua directory is `/opt/sensor/senso
 
 In addition to the [default Suricata ruleset](https://github.com/OISF/suricata/tree/master/rules) and [Emerging Threads Open ruleset](https://rules.emergingthreats.net/open/), users may provide custom rules files for use by Suricata in Malcolm.
 
-Suricata rules files (with the `*.rules` extension) may be placed in the `./suricata/rules/` subdirectory in the Malcolm installation directory. These new rules files will be picked up immediately for subsequent [PCAP upload](upload.md#Upload), and for [live analysis](live-analysis.md#LocalPCAP) will be applied by either restarting Malcolm or when the [automatic rule update process](https://suricata-update.readthedocs.io/en/latest/) runs (if automatic rule updates are enabled). This can also be done manually without completely restarting Malcolm by running the following commands from the Malcolm installation directory:
+Suricata rules files (with the `*.rules` extension) may be placed in the `./suricata/rules/` subdirectory in the Malcolm installation directory. These new rules files will be picked up immediately for subsequent [PCAP upload](upload.md#Upload), and for [live analysis](live-analysis.md#LocalPCAP) will be applied by restarting Malcolm. This can also be done manually without interrupting the Suricata processes by running the following commands from the Malcolm installation directory.
 
+First, for the `suricata-live` container:
+
+```bash
+$ docker compose exec -u $(id -u) suricata-live bash -c 'suricata_config_populate.py --suricata /usr/bin/suricata-offline && kill -USR2 $(pidof suricata)'
 ```
-docker compose exec suricata-live supervisorctl restart live-suricata
+
+Then, for the `suricata` container:
+
+```bash
+$ docker compose exec -u $(id -u) suricata bash -c 'suricata_config_populate.py --suricata /usr/bin/suricata-offline && kill -USR2 $(pidof suricata-offline)'
 ```
 
+Alternately, both Suricata services could be completely restarted with `./scripts/restart -s suricata suricata-live`.
+
+For [Kubernetes deployments of Malcolm](kubernetes.md#Kubernetes), recreating the `suricata-offline-custom-rules-volume` and `suricata-live-custom-rules-volume` configMaps used by the [`suricata`]({{ site.github.repository_url }}/blob/{{ site.github.build_revision }}/kubernetes/11-suricata.yml) and [`suricata-live`]({{ site.github.repository_url }}/blob/{{ site.github.build_revision }}/kubernetes/22-suricata-live.yml) containers, respectively, and restarting those containers, will cause changes to custom rules files to be applied.
+
 If the `SURICATA_CUSTOM_RULES_ONLY` [environment variable](malcolm-config.md#MalcolmConfigEnvVars) is set to `true`, Malcolm will bypass the default Suricata rulesets and use only the user-defined rules.
 
 On [Hedgehog Linux](hedgehog.md), the Suricata custom rules directory is `/opt/sensor/sensor_ctl/suricata/rules/`, and the `SURICATA_CUSTOM_RULES_ONLY` environment variable can be found in [`/opt/sensor/sensor_ctl/control_vars.conf`]({{ site.github.repository_url }}/blob/{{ site.github.build_revision }}/hedgehog-iso/interface/sensor_ctl/control_vars.conf). New rules can be applied by restarting capture processes:

docs/malcolm-hedgehog-e2e-iso-install.md

@@ -4,7 +4,7 @@ This document outlines how to install [Malcolm]({{ site.github.repository_url }}
 
 The Malcolm and Hedgehog Linux installers as described in these instructions are intended to be used to **replace** the existing operating system (if any) of the respective systems onto which they are installed; and, as such, are designed to require as little user input as possible. For this reason, there are NO user prompts or confirmations about partitioning and reformatting hard disks for use by the operating system. The installer assumes that all non-removable storage media (eg., SSD, HDD, NVMe, etc.) are available for use and ⛔🆘😭💀 ***will partition and format them without warning*** 💀😭🆘⛔.
 
-In contrast to using the ISO installer, Malcolm can also be installed on any x86-64 (also known as x64, x86_64, AMD64, and Intel 64) or AArch64 (also known as ARM64) platform capable of running Docker or Podman. See the [installation example using Ubuntu 22.04 LTS](ubuntu-install-example.md#InstallationExample) for that method of installation and configuration, or [Windows host system configuration](host-config-windows.md#HostSystemConfigWindows) and [macOS host system configuration](host-config-macos.md#HostSystemConfigMac) for those platforms.
+In contrast to using the ISO installer, Malcolm can also be installed on any x86-64 (also known as x64, x86_64, AMD64, and Intel 64) or AArch64 (also known as ARM64) platform capable of running Docker or Podman. See the [installation example using Ubuntu 24.04 LTS](ubuntu-install-example.md#InstallationExample) for that method of installation and configuration, or [Windows host system configuration](host-config-windows.md#HostSystemConfigWindows) and [macOS host system configuration](host-config-macos.md#HostSystemConfigMac) for those platforms.
 
 ### <a name="TableOfContents"></a> Table of Contents
 

docs/quickstart.md

@@ -6,7 +6,7 @@
 
 ## <a name="GetMalcolm"></a>Getting Malcolm
 
-For a `TL;DR` example of downloading, configuring, and running Malcolm in Docker on a Linux platform, see **[Installation example using Ubuntu 22.04 LTS](ubuntu-install-example.md#InstallationExample)**.
+For a `TL;DR` example of downloading, configuring, and running Malcolm in Docker on a Linux platform, see **[Installation example using Ubuntu 24.04 LTS](ubuntu-install-example.md#InstallationExample)**.
 
 For a more in-depth guide convering installing both Malcolm and a [Hedgehog Linux](hedgehog.md) sensor using the [Malcolm installer ISO](malcolm-iso.md#ISO) and [Hedgehog Linux installer ISO](hedgehog-installation.md#HedgehogInstallation), see **[End-to-end Malcolm and Hedgehog Linux ISO Installation](malcolm-hedgehog-e2e-iso-install.md#InstallationExample)**.