-
Notifications
You must be signed in to change notification settings - Fork 87
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This commit add a "PIDONE Validated Architecture" composed by: - 3 x ocp masters - 3 x ocp workers - 1 x testOperator-tainted ocp worker The additional workers are required so that we can perform disruptive tests simulating ocp failures (reboots/crashes/etc). Similarly, we need a special worker that would only run testoperator pods so that the tests will not be affected by the simulated infrastructure failures.
- Loading branch information
Showing
20 changed files
with
1,461 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| --- | ||
| vas: | ||
| pidone: | ||
| stages: | ||
| - pre_stage_run: | ||
| - name: Apply taint on worker-3 | ||
| type: cr | ||
| definition: | ||
| spec: | ||
| taints: | ||
| - effect: NoSchedule | ||
| key: testOperator | ||
| value: 'true' | ||
| - effect: NoExecute | ||
| key: testOperator | ||
| value: 'true' | ||
| kind: Node | ||
| resource_name: worker-3 | ||
| state: patched | ||
| path: examples/va/pidone/nncp | ||
| wait_conditions: | ||
| - >- | ||
| oc -n openstack wait nncp | ||
| -l osp/nncm-config-type=standard | ||
| --for jsonpath='{.status.conditions[0].reason}'=SuccessfullyConfigured | ||
| --timeout=180s | ||
| values: | ||
| - name: network-values | ||
| src_file: values.yaml | ||
| build_output: nncp.yaml | ||
|
|
||
| - path: examples/va/pidone | ||
| wait_conditions: | ||
| - >- | ||
| oc -n openstack wait osctlplane controlplane --for condition=Ready | ||
| --timeout=30m | ||
| values: | ||
| - name: network-values | ||
| src_file: nncp/values.yaml | ||
| - name: service-values | ||
| src_file: service-values.yaml | ||
| build_output: control-plane.yaml | ||
|
|
||
| - path: examples/va/pidone/edpm/nodeset | ||
| wait_conditions: | ||
| - >- | ||
| oc -n openstack wait openstackdataplanenodeset | ||
| openstack-edpm | ||
| --for condition=SetupReady | ||
| --timeout=600s | ||
| values: | ||
| - name: edpm-nodeset-values | ||
| src_file: values.yaml | ||
| build_output: nodeset.yaml | ||
|
|
||
| - path: examples/va/pidone/edpm/deployment | ||
| wait_conditions: | ||
| - >- | ||
| oc -n openstack wait openstackdataplanedeployment | ||
| edpm-deployment | ||
| --for condition=Ready | ||
| --timeout=40m | ||
| values: | ||
| - name: edpm-deployment-values | ||
| src_file: values.yaml | ||
| build_output: deployment.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| # OpenStack on Highly Available OpenShift Cluster | ||
|
|
||
| This is a collection of CR templates that represent a validated Red Hat OpenStack Services on OpenShift deployment that has the following characteristics: | ||
|
|
||
| - 3 masters plus 3 workers OpenShift cluster | ||
| - 1 worker dedicated to run testOperator taint-tolerant pods, see [test-operator](https://github.com/openstack-k8s-operators/test-operator) | ||
| - 3-replica Galera database | ||
| - 3-replica RabbitMQ | ||
| - OVN networking | ||
| - Network isolation over a single NIC | ||
| - 3 compute nodes | ||
| - Swift enabled and used as Glance backend | ||
|
|
||
|
|
||
| # Configuring networking and deploy the OpenStack control plane | ||
|
|
||
| ## Assumptions | ||
|
|
||
| - A storage class called `local-storage` should already exist. | ||
|
|
||
| ## Initialize | ||
|
|
||
| Switch to the "openstack" namespace | ||
| ``` | ||
| oc project openstack | ||
| ``` | ||
| Change to the pidone directory | ||
| ``` | ||
| cd architecture/examples/va/pidone | ||
| ``` | ||
| Edit the [nncp/values.yaml](nncp/values.yaml) and | ||
| [service-values.yaml](service-values.yaml) files to suit | ||
| your environment. | ||
| ``` | ||
| vi nncp/values.yaml | ||
| vi service-values.yaml | ||
| ``` | ||
|
|
||
| ## Apply node network configuration | ||
|
|
||
| Generate the node network configuration | ||
| ``` | ||
| kustomize build nncp > nncp.yaml | ||
| ``` | ||
| Apply the NNCP CRs | ||
| ``` | ||
| oc apply -f nncp.yaml | ||
| ``` | ||
| Wait for NNCPs to be available | ||
| ``` | ||
| oc wait nncp -l osp/nncm-config-type=standard --for jsonpath='{.status.conditions[0].reason}'=SuccessfullyConfigured --timeout=300s | ||
| ``` | ||
|
|
||
| ## Apply networking and control-plane configuration | ||
|
|
||
| Generate the control-plane and networking CRs. | ||
| ``` | ||
| kustomize build > control-plane.yaml | ||
| ``` | ||
| Apply the CRs | ||
| ``` | ||
| oc apply -f control-plane.yaml | ||
| ``` | ||
|
|
||
| Wait for control plane to be available | ||
| ``` | ||
| oc wait osctlplane controlplane --for condition=Ready --timeout=600s | ||
| ``` | ||
|
|
||
| ## Apply dataplane nodeset and deployment configuration | ||
|
|
||
| Generate the dataplane nodeset CR. | ||
| ``` | ||
| kustomize build edpm/nodeset > nodeset.yaml | ||
| ``` | ||
| Apply the CR | ||
| ``` | ||
| oc apply -f nodeset.yaml | ||
| ``` | ||
|
|
||
| Wait for the nodeset to reach the SetupReady condition | ||
| ``` | ||
| oc -n openstack wait openstackdataplanenodeset openstack-edpm --for condition=SetupReady --timeout=600s | ||
| ``` | ||
|
|
||
| Generate the dataplane deployment CR. | ||
| ``` | ||
| kustomize build edpm/deployment > deployment.yaml | ||
| ``` | ||
| Apply the CR | ||
| ``` | ||
| oc apply -f deployment.yaml | ||
| ``` | ||
|
|
||
| Wait for the dataplanedeployment to reach the "Ready" condition | ||
| ``` | ||
| oc -n openstack wait openstackdataplanedeployment edpm-deploymenti --for condition=Ready --timeout=40m | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| --- | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
|
|
||
| components: | ||
| - ../../../../../va/pidone/edpm/deployment | ||
|
|
||
| resources: | ||
| - values.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # yamllint disable rule:line-length | ||
| # local-config: referenced, but not emitted by kustomize | ||
| --- | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: edpm-deployment-values | ||
| annotations: | ||
| config.kubernetes.io/local-config: "true" | ||
| data: | ||
| deployment: | ||
| name: edpm-deployment |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| --- | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
|
|
||
| components: | ||
| - ../../../../../va/pidone/edpm/nodeset | ||
|
|
||
| resources: | ||
| - values.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,172 @@ | ||
| # yamllint disable rule:line-length | ||
| # local-config: referenced, but not emitted by kustomize | ||
| --- | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: edpm-nodeset-values | ||
| annotations: | ||
| config.kubernetes.io/local-config: "true" | ||
| data: | ||
| ssh_keys: | ||
| # Authorized keys that will have access to the dataplane computes via SSH | ||
| authorized: CHANGEME | ||
| # The private key that will have access to the dataplane computes via SSH | ||
| private: CHANGEME2 | ||
| # The public key that will have access to the dataplane computes via SSH | ||
| public: CHANGEME3 | ||
| nodeset: | ||
| ansible: | ||
| ansibleUser: cloud-admin | ||
| ansiblePort: 22 | ||
| ansibleVars: | ||
| timesync_ntp_servers: | ||
| - hostname: clock.redhat.com | ||
| # CHANGEME -- see https://access.redhat.com/solutions/253273 | ||
| # edpm_bootstrap_command: | | ||
| # subscription-manager register --username <subscription_manager_username> \ | ||
| # --password <subscription_manager_password> | ||
| # podman login -u <registry_username> -p <registry_password> registry.redhat.io | ||
| edpm_network_config_hide_sensitive_logs: false | ||
| edpm_network_config_os_net_config_mappings: | ||
| edpm-compute-0: | ||
| nic2: 6a:fe:54:3f:8a:02 # CHANGEME | ||
| edpm-compute-1: | ||
| nic2: 6b:fe:54:3f:8a:02 # CHANGEME | ||
| edpm-compute-2: | ||
| nic2: 6c:fe:54:3f:8a:02 # CHANGEME | ||
| edpm_network_config_template: | | ||
| --- | ||
| {% set mtu_list = [ctlplane_mtu] %} | ||
| {% for network in nodeset_networks %} | ||
| {{ mtu_list.append(lookup('vars', networks_lower[network] ~ '_mtu')) }} | ||
| {%- endfor %} | ||
| {% set min_viable_mtu = mtu_list | max %} | ||
| network_config: | ||
| - type: interface | ||
| name: nic1 | ||
| use_dhcp: true | ||
| mtu: {{ min_viable_mtu }} | ||
| - type: ovs_bridge | ||
| name: {{ neutron_physical_bridge_name }} | ||
| mtu: {{ min_viable_mtu }} | ||
| use_dhcp: false | ||
| dns_servers: {{ ctlplane_dns_nameservers }} | ||
| domain: {{ dns_search_domains }} | ||
| addresses: | ||
| - ip_netmask: {{ ctlplane_ip }}/{{ ctlplane_cidr }} | ||
| routes: {{ ctlplane_host_routes }} | ||
| members: | ||
| - type: interface | ||
| name: nic2 | ||
| mtu: {{ min_viable_mtu }} | ||
| # force the MAC address of the bridge to this interface | ||
| primary: true | ||
| {% for network in nodeset_networks %} | ||
| - type: vlan | ||
| mtu: {{ lookup('vars', networks_lower[network] ~ '_mtu') }} | ||
| vlan_id: {{ lookup('vars', networks_lower[network] ~ '_vlan_id') }} | ||
| addresses: | ||
| - ip_netmask: | ||
| {{ lookup('vars', networks_lower[network] ~ '_ip') }}/{{ lookup('vars', networks_lower[network] ~ '_cidr') }} | ||
| routes: {{ lookup('vars', networks_lower[network] ~ '_host_routes') }} | ||
| {% endfor %} | ||
| edpm_nodes_validation_validate_controllers_icmp: false | ||
| edpm_nodes_validation_validate_gateway_icmp: false | ||
| edpm_selinux_mode: enforcing | ||
| edpm_sshd_allowed_ranges: | ||
| - 192.168.122.0/24 | ||
| edpm_sshd_configure_firewall: true | ||
| enable_debug: false | ||
| gather_facts: false | ||
| neutron_physical_bridge_name: br-ex | ||
| neutron_public_interface_name: eth0 | ||
| service_net_map: | ||
| nova_api_network: internalapi | ||
| nova_libvirt_network: internalapi | ||
| storage_mtu: 9000 | ||
| storage_mgmt_mtu: 9000 | ||
| storage_mgmt_vlan_id: 23 | ||
| storage_mgmt_cidr: "24" | ||
| storage_mgmt_host_routes: [] | ||
| networks: | ||
| - defaultRoute: true | ||
| name: ctlplane | ||
| subnetName: subnet1 | ||
| - name: internalapi | ||
| subnetName: subnet1 | ||
| - name: storage | ||
| subnetName: subnet1 | ||
| - name: tenant | ||
| subnetName: subnet1 | ||
| nodes: | ||
| edpm-compute-0: | ||
| ansible: | ||
| ansibleHost: 192.168.122.100 | ||
| hostName: edpm-compute-0 | ||
| networks: | ||
| - defaultRoute: true | ||
| fixedIP: 192.168.122.100 | ||
| name: ctlplane | ||
| subnetName: subnet1 | ||
| - name: internalapi | ||
| subnetName: subnet1 | ||
| - name: storage | ||
| subnetName: subnet1 | ||
| - name: storagemgmt | ||
| subnetName: subnet1 | ||
| - name: tenant | ||
| subnetName: subnet1 | ||
| edpm-compute-1: | ||
| ansible: | ||
| ansibleHost: 192.168.122.101 | ||
| hostName: edpm-compute-1 | ||
| networks: | ||
| - defaultRoute: true | ||
| fixedIP: 192.168.122.101 | ||
| name: ctlplane | ||
| subnetName: subnet1 | ||
| - name: internalapi | ||
| subnetName: subnet1 | ||
| - name: storage | ||
| subnetName: subnet1 | ||
| - name: storagemgmt | ||
| subnetName: subnet1 | ||
| - name: tenant | ||
| subnetName: subnet1 | ||
| edpm-compute-2: | ||
| ansible: | ||
| ansibleHost: 192.168.122.102 | ||
| hostName: edpm-compute-2 | ||
| networks: | ||
| - defaultRoute: true | ||
| fixedIP: 192.168.122.102 | ||
| name: ctlplane | ||
| subnetName: subnet1 | ||
| - name: internalapi | ||
| subnetName: subnet1 | ||
| - name: storage | ||
| subnetName: subnet1 | ||
| - name: storagemgmt | ||
| subnetName: subnet1 | ||
| - name: tenant | ||
| subnetName: subnet1 | ||
| services: | ||
| - bootstrap | ||
| - configure-network | ||
| - validate-network | ||
| - install-os | ||
| - configure-os | ||
| - ssh-known-hosts | ||
| - run-os | ||
| - reboot-os | ||
| - install-certs | ||
| - ovn | ||
| - neutron-metadata | ||
| - libvirt | ||
| - nova | ||
| nova: | ||
| migration: | ||
| ssh_keys: | ||
| private: CHANGEME4 | ||
| public: CHANGEME5 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| --- | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
|
|
||
| components: | ||
| - ../../../va/pidone | ||
|
|
||
| resources: | ||
| - nncp/values.yaml | ||
| - service-values.yaml |
Oops, something went wrong.