Skip to content

Commit

Permalink
docs: add runs and resource concepts
Browse files Browse the repository at this point in the history
  • Loading branch information
ffforest committed Jan 22, 2025
1 parent 3222789 commit f7fc4dd
Show file tree
Hide file tree
Showing 10 changed files with 52 additions and 12 deletions.
40 changes: 38 additions & 2 deletions docs/kusion/3-concepts/10-resources.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,43 @@
# Resources

Resources are persisted by Kusion server.
Kusion uses [Spec](./6-spec.md) to manage resource specifications. A Kusion resource is a logical concept that encapsulates physical resources on different resource planes, including but not limited to Kubernetes, AWS, AliCloud, Azure and Google Cloud.

Kusion Resources are produced by [Kusion Module Generators](./3-module/1-overview.md) and usually map to a physical resource that can be applied via a Kusion Runtime.

## Runtimes

Runtimes is the consumer of the resources in a Spec by turning them into actual physical resources in infrastructure providers.

Currently there are two in-tree runtimes defined in the Kusion source code but we are planning to make them out-of-tree in the future:

- Kubernetes, used to manage resources inside a Kubernetes cluster. This can be any native Kubernetes resources or CRDs (if you wish to manage infrastructures via [CrossPlane](https://www.crossplane.io/) or [Kubevela](https://kubevela.io/), for example, this is completely doable by creating a Kusion Module with a generator that produces the resource YAML for the relevant CRDs)
- Terraform, used to manage infrastructure resources outside a Kubernetes cluster. Kusion uses Terraform as an executor that can manage basically any infrastructure given there is a [terraform provider](https://developer.hashicorp.com/terraform/language/providers) tailored for the infrastructure API. This is generally used to manage the lifecycle of infrastructure resources on clouds, no matter public or on-prem.

## Resource Planes

Resource Plane is a property of a Kusion resource. It represents the actual plane on which the resource exists. Current resource planes include `kubernetes`,`aws`,`azure`,`google`,`alicloud`,`ant` and `custom`.

## Resource ID, Resource URN and Cloud Resource ID

Kusion Resource ID is a unique identifier for a Kusion Resource within a Spec. It must be unique across a Spec. The resource ID is technically generated by module generators so there are no definite rules for producing a Kusion Resource ID. The best practice is to use the `KubernetesResourceID()` and `TerraformResourceID()` method from [kusion-module-framework](https://github.com/KusionStack/kusion-module-framework) to manage Kusion Resource IDs. You can use the [official module generators](https://github.com/KusionStack/catalog/blob/main/modules/mysql/src/alicloud_rds.go#L164) as a reference.

'''tip
Resource ID validations do exist.
For Kubernetes resources, the resource ID must include API version, kind, namespace (if applicable) and name.
For Terraform resources, the resource ID must include provider namespace, provider name, resource type and resource name.
It's always recommended to use the `KubernetesResourceID()` and `TerraformResourceID()` method from [kusion-module-framework](https://github.com/KusionStack/kusion-module-framework) to produce the Resource IDs.
'''

Kusion Resource URN is used to uniquely identify a Kusion Resource across a Kusion server instance. It consists of `${project-name}:${stack-name}:${workspace-name}:${kusion-resource-id}` to ensure global uniqueness.

Cloud Resource ID is used to map to an actual resource on the cloud. For AWS and Alicloud, this is usually known as the resource `ARN` on the cloud. For Azure and Google Cloud, this is known as the Resource ID. It can be empty in some cases, for example, a Kubernetes resource does not have cloud resource ID.

## Resource Graphs

More to come...
A Resource Graph visualizes the relationship between all resources for a given stack. In the Kusion developer portal, you can inspect the resource graph by clicking on the `Resource Graph` tab on the stack page:

![app-resource-graph](/img/docs/concept/app-resource-graph.png)

You can closely inspect the resource details by hovering over the resource node on the graph.

![app-resource-detail](/img/docs/concept/app-resource-graph-detail.png)
2 changes: 1 addition & 1 deletion docs/kusion/3-concepts/4-workspace/1-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ The `importedResources` block is designed to declare the import of existing clou

### secretStore

The `secretStore` field can be used to access the sensitive data stored in a cloud-based secrets manager. More details can be found in [here](../5-user-guides/1-using-kusion-cli/4-secrets-management/1-using-cloud-secrets.md).
The `secretStore` field can be used to access the sensitive data stored in a cloud-based secrets manager. More details can be found in [here](../../5-user-guides/1-using-kusion-cli/4-secrets-management/1-using-cloud-secrets.md).

### context

Expand Down
12 changes: 8 additions & 4 deletions docs/kusion/3-concepts/9-runs.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,21 @@ A `Run` represents an operation performed on a given [Stack](./2-stack/1-overvie
The APIs to manage runs can be found in the swagger docs under `{server-endpoint}/docs/index.html#/run`.

:::tip
Please note that in the case of APIs the runs APIs are asynchronous. An external system interacting with runs is expected to query the status of runs on a polling basis.
Please note that in the case of APIs the runs APIs are asynchronous. An external system interacting with runs API is expected to query the status of runs on a polling basis.
:::

## Types of Runs

There are 4 types of runs:

- Generate: Generate the resource-facing desired state, or [Spec](./6-spec.md) for the given stack and workspace
- Preview: Preview the resource changes for the given stack and workspace based on the desired state
- Preview: Preview the resource changes for the given stack and workspace based on the difference between desired state and current state
- Apply: Apply the desired state to the target workspace
- Destroy: Destroy the resources for the given stack in the target workspace

## Run History

- Run history are persisted in Kusion Server. To check all runs for a given stack, click on the `Project` tab, select a project and click on the stack tab.
- Run history are persisted in Kusion Server. To check all runs for a given stack, click on the `Project` tab, select a project, click on the stack tab, then select the Runs tab to locate all runs for this stack.

![run-history](/img/docs/concept/run-history.png)

Expand All @@ -43,4 +43,8 @@ The run result may differ based on the different types of runs:

- `Apply` and `Destroy` runs produces a message that says "apply/destroy completed".

The run logs are used for troubleshooting in case of any errors during the run.
The run logs are used for troubleshooting in case of any errors during the run.

## Exception Handling

Kusion workflow includes the execution of out-of-tree go-plugins in the form of [Kusion Module](../3-concepts/3-module/1-overview.md) Generators. If any of the plugin code causes a panic, the stack trace is expected to be printed in the run log for closer examination.
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ kusion workspace show
The `workspace.yaml` is a sample configuration file for workspace management, including `MySQL` module configs. Workspace configurations are usually declared by **Platform Engineers** and will take effect through the corresponding stack.

:::info
More details about the configuration of Workspace can be found in [Concepts of Workspace](../../3-concepts/4-workspace/1-overview.md).
More details about the configuration of Workspace can be found in [Concepts of Workspace](../../../3-concepts/4-workspace/1-overview.md).
:::

## Create Project And Stack
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ This guide is to deploy an app using Kusion, relying on the Kusion CLI and an ex

### Initializing workspace configuration

In version 0.10.0, we have introduced the new concept of [workspaces](../../3-concepts/4-workspace/1-overview.md), which is a logical layer whose configurations represent an opinionated set of defaults, often appointed by the platform team. In most cases workspaces are represented with an "environment" in traditional SDLC terms. These workspaces provide a means to separate the concerns between the **application developers** who wish to focus on business logic, and a group of **platform engineers** who wish to standardize the applications on the platform.
In version 0.10.0, we have introduced the new concept of [workspaces](../../../3-concepts/4-workspace/1-overview.md), which is a logical layer whose configurations represent an opinionated set of defaults, often appointed by the platform team. In most cases workspaces are represented with an "environment" in traditional SDLC terms. These workspaces provide a means to separate the concerns between the **application developers** who wish to focus on business logic, and a group of **platform engineers** who wish to standardize the applications on the platform.

Driven by the discipline of Platform Engineering, management of the workspaces, including create/updating/deleting workspaces and their configurations should be done by dedicated platform engineers in a large software organizations to facilitate a more mature and scalable collaboration pattern.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ If you are running a non-local cluster, you can try to expose it via another way

## Setting up workspace configs

Since v0.10.0, we have introduced the concept of [workspaces](../../3-concepts/4-workspace/1-overview.md), whose configurations represent the part of the application behaviors that platform teams are interested in standardizing, or the ones to eliminate from developer's mind to make their lives easier.
Since v0.10.0, we have introduced the concept of [workspaces](../../../3-concepts/4-workspace/1-overview.md), whose configurations represent the part of the application behaviors that platform teams are interested in standardizing, or the ones to eliminate from developer's mind to make their lives easier.

In the case of setting up Prometheus, there are a few things to set up on the workspace level:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ export AWS_SECRET_ACCESS_KEY="oE/xxxx" # replace it with your SecretKey

## Setting up workspace

Since v0.10.0, we have introduced the concept of [workspaces](../../3-concepts/4-workspace/1-overview.md), whose configurations represent the part of the application behaviors that platform teams are interested in standardizing, or the ones to eliminate from developer's mind to make their lives easier.
Since v0.10.0, we have introduced the concept of [workspaces](../../../3-concepts/4-workspace/1-overview.md), whose configurations represent the part of the application behaviors that platform teams are interested in standardizing, or the ones to eliminate from developer's mind to make their lives easier.

In the case of setting up cloud secrets manager, platform teams need to specify which secrets management solution to use and necessary information to access on the workspace level.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ id: collaborate-with-github-actions

In this article, we will introduce how to use Kusion CLI in combination with GitHub Actions to achieve team collaboration in production practice.

Adopting the concept of separation of concerns, we divide the staff involved in application delivery and operation into two groups: **Platform Engineers (PEs)** and **Developers (Devs)**. As the builders of the Internal Developer Platform (IDP), platform engineers are primarily responsible for creating the [storage backend](../../3-concepts/7-backend/1-overview.md) for the Kusion CLI in team collaborative scenarios (e.g. AWS S3 or Alicloud OSS), developing custom reusable [Kusion modules](../../3-concepts/3-module/1-overview.md), and creating and maintaining standardized platform configurations in [workspace](../../3-concepts/4-workspace/1-overview.md). While application developers can focus on writing the application business logic and the configuration codes, self-serving the application delivery and operation by triggering the automated CI/CD pipelines. [GitHub Actions](https://github.com/features/actions) is such a CI/CD platform, and by customizing [GitHub Actions workflow](https://docs.github.com/en/actions/using-workflows), the pipeline such as building, testing, and deploying will be executed automatically.
Adopting the concept of separation of concerns, we divide the staff involved in application delivery and operation into two groups: **Platform Engineers (PEs)** and **Developers (Devs)**. As the builders of the Internal Developer Platform (IDP), platform engineers are primarily responsible for creating the [storage backend](../../../3-concepts/7-backend/1-overview.md) for the Kusion CLI in team collaborative scenarios (e.g. AWS S3 or Alicloud OSS), developing custom reusable [Kusion modules](../../../3-concepts/3-module/1-overview.md), and creating and maintaining standardized platform configurations in [workspace](../../../3-concepts/4-workspace/1-overview.md). While application developers can focus on writing the application business logic and the configuration codes, self-serving the application delivery and operation by triggering the automated CI/CD pipelines. [GitHub Actions](https://github.com/features/actions) is such a CI/CD platform, and by customizing [GitHub Actions workflow](https://docs.github.com/en/actions/using-workflows), the pipeline such as building, testing, and deploying will be executed automatically.

In the following sections, we will demonstrate the specific workflow from the perspectives of both PEs and Devs with the sample workflows from our [konfg](https://github.com/KusionStack/konfig) and [catalog](https://github.com/KusionStack/catalog) repository.

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added static/img/docs/concept/app-resource-graph.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit f7fc4dd

Please sign in to comment.