Add metal-api gRPC certificates for new machine allocation (#5)

metal-stack · Aug 6, 2020 · c0da4cd · c0da4cd
1 parent fb3f338
commit c0da4cd
Show file tree

Hide file tree

Showing 18 changed files with 161 additions and 30 deletions.
diff --git a/control-plane/roles/metal/README.md b/control-plane/roles/metal/README.md
@@ -50,32 +50,36 @@ You can look up all the default values of this role [here](defaults/main/main.ya
 
 ### metal-api
 
-| Name                          | Mandatory | Description                                                          |
-| ----------------------------- | --------- | -------------------------------------------------------------------- |
-| metal_api_replicas            |           | The number of deployed replicas of the metal-api                     |
-| metal_api_base_path           |           | The base path of the HTTP server                                     |
-| metal_api_dex_address         |           | The URL to the dex server                                            |
-| metal_api_db_address          |           | The URL of the metal-db                                              |
-| metal_api_db_password         |           | The password of the metal-db                                         |
-| metal_api_ipam_db_address     |           | The URL to the ipam database                                         |
-| metal_api_ipam_db_port        |           | The port of the ipam database                                        |
-| metal_api_ipam_db_name        |           | The database name of the ipam database                               |
-| metal_api_ipam_db_user        |           | The user of the ipam database                                        |
-| metal_api_ipam_db_password    |           | The password of the ipam database                                    |
-| metal_api_nsq_lookupd_address |           | The http address of nsq-lookupd                                      |
-| metal_api_nsq_tcp_address     |           | The tcp address of nsqd                                              |
-| metal_api_nsq_http_address    |           | The http address of nsqd (only used for in-cluster traffic)          |
-| metal_api_nsq_tls_enabled     |           | Enables TLS for nsq                                                  |
-| metal_api_nsq_tls_secret_name |           | The name of the secret where nsq certificates are stored             |
-| metal_api_view_key            |           | The HMAC view key of the metal-api used for API technical access     |
-| metal_api_edit_key            |           | The HMAC edit key of the metal-api used for API technical access     |
-| metal_api_admin_key           |           | The HMAC admin key of the metal-api used for API technical access    |
-| metal_api_sizes               |           | Creates sizes (as masterdata) to the metal-api after deployment      |
-| metal_api_images              |           | Creates images (as masterdata) to the metal-api after deployment     |
-| metal_api_partitions          |           | Creates partitions (as masterdata) to the metal-api after deployment |
-| metal_api_networks            |           | Creates networks (as masterdata) to the metal-api after deployment   |
-| metal_api_ips                 |           | Creates ips (as masterdata) to the metal-api after deployment        |
-| metal_api_resources           |           | Sets the given container resources                                   |
+| Name                             | Mandatory | Description                                                          |
+| -------------------------------- | --------- | -------------------------------------------------------------------- |
+| metal_api_replicas               |           | The number of deployed replicas of the metal-api                     |
+| metal_api_base_path              |           | The base path of the HTTP server                                     |
+| metal_api_dex_address            |           | The URL to the dex server                                            |
+| metal_api_db_address             |           | The URL of the metal-db                                              |
+| metal_api_db_password            |           | The password of the metal-db                                         |
+| metal_api_ipam_db_address        |           | The URL to the ipam database                                         |
+| metal_api_ipam_db_port           |           | The port of the ipam database                                        |
+| metal_api_ipam_db_name           |           | The database name of the ipam database                               |
+| metal_api_ipam_db_user           |           | The user of the ipam database                                        |
+| metal_api_ipam_db_password       |           | The password of the ipam database                                    |
+| metal_api_nsq_lookupd_address    |           | The http address of nsqlookupd (only used for in-cluster traffic)    |
+| metal_api_nsq_tcp_address        |           | The tcp address of nsqd                                              |
+| metal_api_nsq_http_address       |           | The http address of nsqd (only used for in-cluster traffic)          |
+| metal_api_nsq_tls_enabled        |           | Enables TLS for nsq                                                  |
+| metal_api_nsq_tls_secret_name    |           | The name of the secret where nsq certificates are stored             |
+| metal_api_grpc_tls_enabled       |           | Enables TLS for gRPC                                                 |
+| metal_api_grpc_certs_server_key  |           | The gRPC client key as a string                                      |
+| metal_api_grpc_certs_server_cert |           | The gRPC client certificate as a string                              |
+| metal_api_grpc_certs_ca_cert     |           | The gRPC ca certificate as a string                                  |
+| metal_api_view_key               |           | The HMAC view key of the metal-api used for API technical access     |
+| metal_api_edit_key               |           | The HMAC edit key of the metal-api used for API technical access     |
+| metal_api_admin_key              |           | The HMAC admin key of the metal-api used for API technical access    |
+| metal_api_sizes                  |           | Creates sizes (as masterdata) to the metal-api after deployment      |
+| metal_api_images                 |           | Creates images (as masterdata) to the metal-api after deployment     |
+| metal_api_partitions             |           | Creates partitions (as masterdata) to the metal-api after deployment |
+| metal_api_networks               |           | Creates networks (as masterdata) to the metal-api after deployment   |
+| metal_api_ips                    |           | Creates ips (as masterdata) to the metal-api after deployment        |
+| metal_api_resources              |           | Sets the given container resources                                   |
 
 ### masterdata-api
 

diff --git a/control-plane/roles/metal/defaults/main/main.yaml b/control-plane/roles/metal/defaults/main/main.yaml
@@ -6,11 +6,12 @@ metal_set_resource_limits: yes
 
 metal_check_api_health_endpoint: https://{{ metal_api_ingress_dns }}{{ metal_api_base_path }}v1/health
 
-metal_log_level: debug
+metal_log_level: info
 metal_log_encoding: json
 
 # service ports
 metal_api_port: 8080
+metal_api_grpc_port: 50051
 metal_api_metrics_port: 2112
 metal_masterdata_api_port: 8443
 metal_masterdata_api_metrics_port: 2113
@@ -28,11 +29,12 @@ metal_api_ipam_db_port: 5432
 metal_api_ipam_db_name: ipam
 metal_api_ipam_db_user: postgres
 metal_api_ipam_db_password: change-me
-metal_api_nsq_lookupd_address: "nsq-lookupd:4161"
 metal_api_nsq_tcp_address: "{{ metal_control_plane_ingress_dns }}:4150"
 metal_api_nsq_http_address: "nsqd:4151"
+metal_api_nsq_lookupd_address: "nsq-lookupd:4161"
 metal_api_nsq_tls_enabled: true
 metal_api_nsq_tls_secret_name: nsqd
+metal_api_grpc_tls_enabled: true
 metal_api_view_key: change-me
 metal_api_edit_key: change-me
 metal_api_admin_key: change-me

diff --git a/control-plane/roles/metal/files/metal-control-plane/templates/metal-api.yaml b/control-plane/roles/metal/files/metal-control-plane/templates/metal-api.yaml
@@ -11,6 +11,17 @@ stringData:
   edit_key: {{ .Values.metal_api.edit_key }}
   admin_key: {{ .Values.metal_api.admin_key }}
 ---
+apiVersion: v1
+kind: Secret
+metadata:
+  labels:
+    app: grpc
+  name: grpc
+data:
+  server_key.pem: {{ .Values.grpc_certs_server_key_enc }}
+  server_cert.pem: {{ .Values.grpc_certs_server_cert_enc }}
+  ca_cert.pem: {{ .Values.grpc_certs_ca_cert_enc }}
+---
 apiVersion: apps/v1
 kind: Deployment
 metadata:
@@ -35,6 +46,8 @@ spec:
         name: metal-api
         ports:
         - containerPort: 8080
+        - containerPort: 50051
+          protocol: TCP
         - containerPort: 2112
         env:
 {{- if eq .Values.zap_level "debug" }}
@@ -93,6 +106,16 @@ spec:
           value: /certs/nsq/ca_cert.pem
         - name: METAL_API_NSQD_CLIENT_CERT_FILE
           value: /certs/nsq/client_cert.pem
+{{- end }}
+        - name: METAL_API_GRPC_TLS_ENABLED
+          value: "{{ .Values.metal_api.grpc.tls.enabled }}"
+{{- if .Values.metal_api.grpc.tls.enabled }}
+        - name: METAL_API_GRPC_CA_CERT_FILE
+          value: /certs/grpc/ca_cert.pem
+        - name: METAL_API_GRPC_SERVER_CERT_FILE
+          value: /certs/grpc/server_cert.pem
+        - name: METAL_API_GRPC_SERVER_KEY_FILE
+          value: /certs/grpc/server_key.pem
 {{- end }}
         - name: METAL_API_MASTERDATA_HMAC
           value: "{{ .Values.masterdata_api.hmac }}"
@@ -115,6 +138,11 @@ spec:
         - name: {{ .Values.metal_api.nsq.tls.secret_name }}
           mountPath: /certs/nsq/
           readOnly: true
+{{- end }}
+{{- if .Values.metal_api.grpc.tls.enabled }}
+        - name: grpc
+          mountPath: /certs/grpc/
+          readOnly: true
 {{- end }}
         - name: masterdata-api-tls
           mountPath: /certs/masterdata-api/
@@ -124,6 +152,11 @@ spec:
       - name: {{ .Values.metal_api.nsq.tls.secret_name }}
         secret:
           secretName: {{ .Values.metal_api.nsq.tls.secret_name }}
+{{- end }}
+{{- if .Values.metal_api.grpc.tls.enabled }}
+      - name: grpc
+        secret:
+          secretName: grpc
 {{- end }}
       - name: masterdata-api-tls
         secret:
@@ -140,6 +173,9 @@ spec:
   - name: "{{ .Values.ports.metal_api }}"
     port: {{ .Values.ports.metal_api }}
     targetPort: 8080
+  - name: "{{ .Values.ports.metal_api_grpc }}"
+    port: {{ .Values.ports.metal_api_grpc }}
+    targetPort: 50051
   - name: "{{ .Values.ports.metal_api_metrics }}"
     port: {{ .Values.ports.metal_api_metrics }}
     targetPort: 2112
@@ -423,11 +459,15 @@ spec:
               value: /certs/nsq/ca_cert.pem
             - name: METAL_API_NSQD_CLIENT_CERT_FILE
               value: /certs/nsq/client_cert.pem
+{{- end }}
             volumeMounts:
+{{- if .Values.metal_api.nsq.tls.enabled }}
             - name: {{ .Values.metal_api.nsq.tls.secret_name }}
               mountPath: /certs/nsq/
               readOnly: true
+{{- end }}
           volumes:
+{{- if .Values.metal_api.nsq.tls.enabled }}
           - name: {{ .Values.metal_api.nsq.tls.secret_name }}
             secret:
               secretName: {{ .Values.metal_api.nsq.tls.secret_name }}

diff --git a/control-plane/roles/metal/files/metal-control-plane/values.yaml b/control-plane/roles/metal/files/metal-control-plane/values.yaml
@@ -43,6 +43,7 @@ resources:
 
 ports:
   metal_api: 8080
+  metal_api_grpc: 50051
   metal_api_metrics: 2112
   masterdata_api: 8443
   masterdata_api_metrics: 2113
@@ -63,13 +64,17 @@ metal_api:
   view_key: ""
   edit_key: ""
   admin_key: ""
+  nsq:
   nsq:
     lookupd_address: ""
     tcp_address: ""
     http_address: ""
     tls:
       enabled: false
       secret_name: ""
+  grpc:
+    tls:
+      enabled: false
   partitions: "[]"
   sizes: "[]"
   images: "[]"

diff --git a/control-plane/roles/metal/tasks/main.yaml b/control-plane/roles/metal/tasks/main.yaml
@@ -33,11 +33,13 @@
     # deployment can take a while due to post install hooks, therefore increasing the timeout for this chart...
     helm_timeout: 600s
     helm_chart_inject_config_hash: yes
+    helm_force: false
 
 - name: Set services for patching ingress controller service exposal
   set_fact:
     metal_tcp_services:
       "5222": "{{ metal_control_plane_namespace }}/metal-console:10001"
+      "50051": "{{ metal_control_plane_namespace }}/metal-api:50051"
     metal_udp_services:
       "5222": "{{ metal_control_plane_namespace }}/metal-console:10001"
   when: metal_expose_ingress_service_ports
@@ -75,6 +77,10 @@
           port: 5222
           protocol: TCP
           targetPort: 5222
+        - name: metal-control-plane-grpc
+          port: 50051
+          protocol: TCP
+          targetPort: 50051
   when: metal_expose_ingress_service_ports
 
 # for automation tests, we need to wait until all the services are ready...

diff --git a/control-plane/roles/metal/templates/metal-values.j2 b/control-plane/roles/metal/templates/metal-values.j2
@@ -34,6 +34,7 @@ metal_console:
 
 ports:
   metal_api: {{ metal_api_port }}
+  metal_api_grpc: {{ metal_api_grpc_port }}
   metal_api_metrics: {{ metal_api_metrics_port }}
   masterdata_api: {{ metal_masterdata_api_port }}
   masterdata_api_metrics: {{ metal_masterdata_api_metrics_port }}
@@ -55,6 +56,9 @@ metal_api:
     tls:
       enabled: {{ metal_api_nsq_tls_enabled }}
       secret_name: "{{ metal_api_nsq_tls_secret_name }}"
+  grpc:
+    tls:
+      enabled: "{{ metal_api_grpc_tls_enabled }}"
   replicas: "{{ metal_api_replicas }}"
   base_path: "{{ metal_api_base_path }}"
   dex_address: "{{ metal_api_dex_address }}"
@@ -130,3 +134,7 @@ mgmtservices: {{ metal_mgmt_services | to_json }}
 ingress_public_dns: "{{ metal_ingress_dns }}"
 deploy_ingress: {{ metal_deploy_ingress }}
 ingress: {{ metal_ingress | to_json }}
+
+grpc_certs_server_key_enc: "{{ metal_api_grpc_certs_server_key | b64encode }}"
+grpc_certs_server_cert_enc: "{{ metal_api_grpc_certs_server_cert | b64encode }}"
+grpc_certs_ca_cert_enc: "{{ metal_api_grpc_certs_ca_cert | b64encode }}"
diff --git a/control-plane/roles/nsq/README.md b/control-plane/roles/nsq/README.md
@@ -30,6 +30,7 @@ You can look up all the default values of this role [here](defaults/main/main.ya
 | nsq_broadcast_address            |           | The nsq broadcast address                                                                                                          |
 | nsq_nsqd_data_size               |           | The size of the nsqd data volume (used when memory cache is full)                                                                  |
 | nsq_tls_enabled                  |           | Enables TLS for nsq                                                                                                                |
-| nsq_certs_client_key             |           | The nsq certifate client key as a string                                                                                           |
+| nsq_certs_client_key             |           | The nsq certificate client key as a string                                                                                           |
 | nsq_certs_client_cert            |           | The nsq client certificate as a string                                                                                             |
 | nsq_certs_ca_cert                |           | The nsq ca certificate as a string                                                                                                 |
+
diff --git a/partition/README.md b/partition/README.md
@@ -19,6 +19,7 @@ You can look up all the default values [here](partition-defaults/main.yaml).
 | metal_partition_timezone                | yes       | The timezone in which this partition is located                                                  |
 | metal_partition_metal_api_addr          | yes       | The address of the metal-api that this partition connects to                                     |
 | metal_partition_metal_api_port          |           | The port of the metal-api that this partition connects to                                        |
+| metal_partition_metal_api_grpc_port     |           | The port of the metal-api gRPC server that this partition connects to                            |
 | metal_partition_metal_api_protocol      |           | The protocol of the metal-api that this partition connects to                                    |
 | metal_partition_metal_api_basepath      |           | The basepath of the metal-api that this partition connects to                                    |
 | metal_partition_metal_api_hmac_edit_key |           | The HMAC edit key used for authenticating at the metal-api                                       |

diff --git a/partition/partition-defaults/main.yaml b/partition/partition-defaults/main.yaml
@@ -5,6 +5,7 @@ metal_partition_id:
 metal_partition_metal_api_protocol: https
 metal_partition_metal_api_addr:
 metal_partition_metal_api_port: 443
+metal_partition_metal_api_grpc_port: 50051
 metal_partition_metal_api_basepath: "/"
 metal_partition_metal_api_hmac_edit_key: change-me
 

diff --git a/partition/roles/docker-on-cumulus/tasks/main.yaml b/partition/roles/docker-on-cumulus/tasks/main.yaml
@@ -7,16 +7,24 @@
       - ca-certificates
       - curl
       - gnupg2
+  when: ansible_distribution_major_version == "3"
 
 - name: setup key for docker
   apt_key:
     url: https://download.docker.com/linux/ubuntu/gpg
     id: 9DC858229FC7DD38854AE2D88D81803C0EBFCD88
 
-- name: add docker engine repo
+- name: add docker engine repo for debian jessie
   apt_repository:
     repo: 'deb [arch=amd64] https://download.docker.com/linux/debian jessie edge'
     update_cache: yes
+  when: ansible_distribution_major_version == "3"
+
+- name: add docker engine repo for debian buster
+  apt_repository:
+    repo: 'deb [arch=amd64] https://download.docker.com/linux/debian buster stable'
+    update_cache: yes
+  when: ansible_distribution_major_version == "4"
 
 - name: install docker
   apt:

diff --git a/partition/roles/ipmi-catcher/defaults/main/global-defaults b/partition/roles/ipmi-catcher/defaults/main/global-defaults
@@ -0,0 +1 @@
+../../../../../defaults
diff --git a/partition/roles/ipmi-catcher/tasks/main.yaml b/partition/roles/ipmi-catcher/tasks/main.yaml
@@ -1,4 +1,7 @@
 ---
+- name: Gather release versions
+  setup_yaml:
+
 - name: Check mandatory variables for this role are set
   assert:
     fail_msg: "not all mandatory variables given, check role documentation"

diff --git a/partition/roles/metal-core/README.md b/partition/roles/metal-core/README.md
@@ -25,6 +25,12 @@ You can look up all the default values of this role [here](defaults/main/main.ya
 | metal_core_change_boot_order              |           |                                                                                                                                                                        |
 | metal_core_reconfigure_switch             |           | If set to true, metal-core will automatically reconfigure files on the switch                                                                                          |
 | metal_core_reconfigure_switch_interval    |           | The interval in which the switch config gets applied from information received from the metal-api                                                                      |
+| metal_core_grpc_address                   |           | The address (host:port) of the metal-api gRPC server                                                                                                                   |
+| metal_core_grpc_tls_enabled               |           | Whether TLS is enabled for the metal-api gRPC server                                                                                                                   |
+| metal_core_grpc_cert_dir                  |           | Path to the gRPC certificate files on the host machine                                                                                                                 |
+| metal_core_grpc_ca_cert                   |           | The gRPC CA certificate content                                                                                                                                        |
+| metal_core_grpc_client_cert               |           | The gRPC client certificate content                                                                                                                                    |
+| metal_core_grpc_client_key                |           | The gRPC client certificate key content                                                                                                                                |
 | metal_core_additional_bridge_vids         |           |                                                                                                                                                                        |
 | metal_core_additional_bridge_ports        |           |                                                                                                                                                                        |
 | metal_core_spine_uplinks                   |           | The switch ports that connect a leaf to a spine switch or other ports that need to be part of the EVPN underlay fabric. Defaults to `swp31` and `swp32` at the metal-core. |