-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #1 from PhilipSchmid/add_non_hostpol_example
Added netpol examples without host policies
- Loading branch information
Showing
35 changed files
with
798 additions
and
45 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| version: 2 | ||
| updates: | ||
| - package-ecosystem: "terraform" | ||
| directory: /deploy | ||
| schedule: | ||
| interval: "daily" | ||
| time: "23:00" | ||
| timezone: "Europe/Zurich" | ||
| open-pull-requests-limit: 3 | ||
| rebase-strategy: "disabled" | ||
| labels: | ||
| - ci/dependabot | ||
| - kind/enhancement |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| name: Validation Actions | ||
| on: | ||
| pull_request_target: | ||
| types: | ||
| - opened | ||
| - synchronize | ||
| - reopened | ||
| push: | ||
| branches: | ||
| - main | ||
| jobs: | ||
| formatting: | ||
| runs-on: ubuntu-22.04 | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 | ||
| with: | ||
| ref: ${{ github.sha }} | ||
| - name: terraform fmt | ||
| uses: dflook/terraform-fmt-check@529e30563b2c558dc0b8c450b5cec1cc93bd7fe4 | ||
| with: | ||
| path: /deploy | ||
| docs: | ||
| runs-on: ubuntu-22.04 | ||
| needs: formatting | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 | ||
| with: | ||
| ref: ${{ github.ref }} | ||
| - name: Render terraform docs inside the README.md and push changes back to PR branch | ||
| uses: terraform-docs/gh-actions@d1c99433f7a1e5003ef213d70f89aaa47cb0b675 | ||
| with: | ||
| working-dir: /deploy | ||
| output-file: README.md | ||
| output-method: inject | ||
| output-format: markdown table | ||
| indention: 3 | ||
| git-push: "true" | ||
| validate-netpol-yamls: | ||
| runs-on: ubuntu-22.04 | ||
| needs: formatting | ||
| steps: | ||
| - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 | ||
| - name: Validate Network Policy YAML files | ||
| run: yamllint netpols/*/*.yaml | ||
| validate-cilium-valuesyaml: | ||
| runs-on: ubuntu-22.04 | ||
| needs: formatting | ||
| steps: | ||
| - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 | ||
| - name: Validate Cilium Helm values YAML file | ||
| run: yamllint deploy/03-cilium-values-1.14.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # https://terraform-docs.io/user-guide/configuration/ | ||
| formatter: "markdown table" | ||
| output: | ||
| file: README.md | ||
| mode: inject | ||
| settings: | ||
| indent: 3 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| # Demo Network Policies (without Host Policies) | ||
|
|
||
| ## Generic Hubble Configuration for Visibility | ||
| ```bash | ||
| # - Since we enabled TLS for Hubble, we need to configure Hubble CLI accordingly | ||
| # - Get Hubble CLI from here: https://docs.cilium.io/en/stable/gettingstarted/hubble_setup/#install-the-hubble-client | ||
| hubble config set tls true | ||
| hubble config set tls-ca-cert-files /path/to/cilium-netpol-demo/deploy/cilium-ca-crt.pem | ||
| hubble config set tls-server-name "*.hubble-relay.cilium.io" | ||
| # Open the port-forwarding in a separate shell: | ||
| kubectl port-forward -n kube-system svc/hubble-relay 4245:443 | ||
| # Finally check Hubble CLI's connection and the flows: | ||
| hubble status | ||
| ``` | ||
|
|
||
| ## Infrastructure Components | ||
| ```bash | ||
| # Label namespaces with Ingress resources that should be reachable from Nginx ingress controller with `exposed=true`: | ||
| kubectl label namespace goldpinger exposed=true | ||
| kubectl label namespace monitoring exposed=true | ||
| kubectl label namespace kube-system exposed=true # Required for Hubble-UI. Move Hubble-UI to a dedicated namespaces for production: https://docs.cilium.io/en/stable/gettingstarted/hubble/#enable-the-hubble-ui ("Helm (Standalone install)" tab) | ||
|
|
||
| # Label namespaces with metric endpoints that should be scraped by Prometheus with `metrics=true`: | ||
| kubectl label namespace goldpinger metrics=true | ||
| kubectl label namespace kube-system metrics=true | ||
| kubectl label namespace ingress-nginx metrics=true | ||
|
|
||
| # Kube-system | ||
| kubectl apply -f cnp-infra-kube-system.yaml | ||
|
|
||
| # Nginx Ingress Controller | ||
| kubectl apply -f cnp-infra-ingress-nginx.yaml | ||
|
|
||
| # Cert-Manager | ||
| kubectl apply -f cnp-infra-cert-manager.yaml | ||
|
|
||
| # Kube Prometheus Stack | ||
| kubectl apply -f cnp-infra-monitoring-stack.yaml | ||
|
|
||
| # Goldpinger (only in case Goldpinger is deployed on the cluster): | ||
| kubectl apply -f cnp-infra-goldpinger.yaml | ||
|
|
||
| # Finally, check for wrongly dropped flows: | ||
| hubble observe -t policy-verdict -f --verdict DROPPED | ||
| ``` | ||
|
|
||
| ## Cluster-wide Policies | ||
| The goal should be to deploy new user workload namespaces with only a very small set of default policies. Hence, you can leverage CiliumClusterwideNetworkPolicies (CCNPs) to predefine permits of common services like ingress or monitoring to communicate with the new namespace. | ||
|
|
||
| In addition, you can leverage CCNPs to enable Cilium's DNS visibility by applying an egress policy that uses `toPorts[*].rules.dns`. | ||
|
|
||
| ```bash | ||
| kubectl apply -f ccnp-global-infra.yaml | ||
| ``` | ||
|
|
||
| ## User Workload | ||
| As there are already CCNPs matching for everything `spec.endpointSelector: {}`, in both directions (`spec.ingress` and `spec.egress`), newly created (user) namespaces are already in a deny-all state. As a result, even namespace internal traffic is denied until a new `allow-within-namespace` CiliumNetworkPolicy (CNP) is created to allow this traffic: | ||
|
|
||
| ```yaml | ||
| --- | ||
| apiVersion: "cilium.io/v2" | ||
| kind: CiliumNetworkPolicy | ||
| metadata: | ||
| name: allow-within-namespace | ||
| namespace: my-namespace-xy | ||
| spec: | ||
| description: Allow NS internal traffic, block everything else | ||
| endpointSelector: {} | ||
| ingress: | ||
| - fromEndpoints: | ||
| - {} | ||
| egress: | ||
| - toEndpoints: | ||
| - {} | ||
| ``` | ||
| Have a look at the following template that could be used for new user workload namespaces. | ||
| - `cnp-user-template.yaml`: | ||
| - Add the namespace label `exposed: "true"` in case Nginx ingress should be able to serve Ingresses from this namespace. | ||
| - Add the namespace label `metrics: "true"` in case Prometheus should be able to scrape metrics endpoints from this namespace. | ||
| - Allow all ingress and egress traffic **within** the namespace | ||
| - Optional: Additional application specific CNPs to explicitly allow connections to and from namespace-external sources/destinations. | ||
|
|
||
| Check out `demo-app-podinfo.sh` to simulate the deployment of new user workload. | ||
|
|
||
| ## Troubleshooting | ||
| To troubleshoot connectivity issues or false positive denies, use Hubble UI and especially Hubble CLI. Hubble can either be directly used within a Cilium agent pod (only sees node local traffic) or in an even more powerful way, via the dedicated [Hubble CLI](https://docs.cilium.io/en/stable/gettingstarted/hubble_setup/#install-the-hubble-client). This Hubble CLI then needs to point to the Hubble-Relay service which aggregates the flows from all Cilium agents / nodes. | ||
|
|
||
| ```bash | ||
| # Temporarily expose the `hubble-relay` ClusterIP service via `kubectl port-forward` (blocking call, separate shell): | ||
| kubectl port-forward -n kube-system svc/hubble-relay 4245:443 | ||
|
|
||
| # Check for dropped traffic: | ||
| hubble observe -t policy-verdict -f --verdict DROPPED | ||
| ``` | ||
|
|
||
| Improve your Hubble CLI outputs even further by using additional filtering constraints (issue `hubble observe --help` to see all available options): | ||
| - `--ip` / `--to-ip` / `--from-ip` | ||
| - `-n` / `--namespace` / `--to-namespace` / `--from-namespace` | ||
| - `--port` / `--to-pod` / `--from-pod` | ||
| - `--node-name` | ||
|
|
||
| ## Sources: | ||
| - https://docs.cilium.io/en/stable/gettingstarted/hubble_setup/#install-the-hubble-client | ||
| - https://docs.cilium.io/en/stable/gettingstarted/hubble_cli/#hubble-cli |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.