Create pre-production stage for CI pipeline #2282

nhatnghiho · 2025-03-20T15:59:58Z

Issues:

Resolves #CryptoAlg-2983 and #CryptoAlg-2984

Description of changes:

This PR implements steps to deploy to pre-production in the CI pipeline, which will act as template for production deployment later on. The PR also sets up a dev deployment environment for testing purposes.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

codecov-commenter · 2025-03-20T16:23:46Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.05%. Comparing base (652f7a9) to head (b93ce94).
Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2282      +/-   ##
==========================================
+ Coverage   79.02%   79.05%   +0.02%     
==========================================
  Files         614      614              
  Lines      106934   107058     +124     
  Branches    15145    15162      +17     
==========================================
+ Hits        84504    84634     +130     
+ Misses      21777    21771       -6     
  Partials      653      653

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

skmcgrail · 2025-03-26T20:34:59Z

tests/ci/cdk/cdk/codebuild/ec2_test_framework_omnibus.yaml

-        image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_clang-7x-bm-framework_latest
+        image: 183295444613.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_clang-7x-bm-framework_latest


Should remain as the team account

skmcgrail · 2025-03-26T20:35:05Z

tests/ci/cdk/cdk/codebuild/ec2_test_framework_omnibus.yaml

-        image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_clang-7x-bm-framework_latest
+        image: 183295444613.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_clang-7x-bm-framework_latest


Should remain as the team account

skmcgrail · 2025-03-26T20:35:13Z

tests/ci/cdk/cdk/codebuild/ec2_test_framework_omnibus.yaml

-        image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_clang-7x-bm-framework_latest
+        image: 183295444613.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_clang-7x-bm-framework_latest


Should remain as the team account

skmcgrail · 2025-03-26T20:35:19Z

tests/ci/cdk/cdk/codebuild/ec2_test_framework_omnibus.yaml

-        image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_clang-7x-bm-framework_latest
+        image: 183295444613.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_clang-7x-bm-framework_latest


Should remain as the team account

skmcgrail · 2025-04-02T18:32:24Z

tests/ci/cdk/cdk/aws_lc_ec2_test_framework_ci_stack.py

+        if env.account == PRE_PROD_ACCOUNT:
+            github_repo_owner = STAGING_GITHUB_REPO_OWNER
+            github_repo_name = STAGING_GITHUB_REPO_NAME


How does this work for deploying to your own personal account if you want to go through setting up the pipeline and all that?

I guess these are correctly set to the values from the pipeline cdk_env in AwsLcCiPipeline?

skmcgrail · 2025-04-02T19:32:44Z

tests/ci/cdk/pipeline/scripts/build_target.sh

+  while [[ ${attempt} -le ${MAX_RETRY} ]]; do
+    if [[ $attempt -gt 0 ]]; then
+      echo "Retrying ${attempt}/${MAX_RETRY}..."
+    fi
+
+    attempt=$((attempt + 1))
+


skmcgrail · 2025-04-02T19:43:17Z

tests/ci/cdk/pipeline/scripts/build_target.sh

+
+assume_role
+
+if [[ -z "${BUILD_TYPE+x}" || -z "${BUILD_TYPE}" ]]; then


Suggested change

if [[ -z "${BUILD_TYPE+x}" || -z "${BUILD_TYPE}" ]]; then

if [[ -z "${BUILD_TYPE:+x}" ]]; then

Does both of these for you. If BUILD_TYPE is either unset / empty string or not defined it will be a empty string substituted for this check. Otherwise x will be substituted and thus not move into the if statement body.

skmcgrail · 2025-04-02T19:43:34Z

tests/ci/cdk/pipeline/scripts/build_target.sh

+fi
+
+if [[ ${BUILD_TYPE} == "docker" ]]; then
+  if [[ -z "${PLATFORM+x}" || -z "${PLATFORM}" ]]; then


Same note as above.

skmcgrail · 2025-04-02T19:43:40Z

tests/ci/cdk/pipeline/scripts/build_target.sh

+fi
+
+if [[ ${BUILD_TYPE} == "ci" ]]; then
+  if [[ -z "${PROJECT+x}" || -z "${PROJECT}" ]]; then


skmcgrail · 2025-04-02T19:44:24Z

tests/ci/cdk/pipeline/scripts/check_trigger_conditions.sh

+if [[ -z "${BUILD_TYPE+x}" || -z "${BUILD_TYPE}" ]]; then
+  echo "No build type provided."
+  exit 1
+fi
+
+if [[ -z "${STACKS+x}" || -z "${STACKS}" ]]; then
+  echo "No stacks provided."
+  exit 1
+fi
+
+if [[ -n "${PREVIOUS_REBUILDS+x}" && -n "${PREVIOUS_REBUILDS}" ]]; then


See previous comments

andrewhop · 2025-04-08T21:41:59Z

tests/ci/cdk/README.md

+### CI Commands
+Use these commands if you wish to deploy individual stacks instead of the entire pipeline.
+
+These commands are run from `aws-lc/tests/ci/cdk`.
+
+If not done previously, bootstrap cdk before running the commands below. Make sure that AWS_ACCOUNT_ID is the AWS account you wish to deploy the CI stacks to.


Is there any reason to keep this support and documented?

This is helpful if we want to deploy 1 stack at a time. Say, if I only want to deploy aws-lc-ci-integration and don't want to deal with anything else, these commands will come in handy.

andrewhop · 2025-04-08T21:58:53Z

tests/ci/cdk/cdk/aws_lc_github_ci_stack.py

+from aws_cdk import (
+    Duration,
+    Stack,
+    aws_codebuild as codebuild,
+    aws_iam as iam,
+    aws_s3_assets,
+    aws_logs as logs,
+    Environment,
+)


In the future it is much easier to review big changes like this if you don't reformat unrelated code. If there wasn't consistent formatting of this cdk code before your change it would much easier to review if one PR is just reformatting everything.

andrewhop · 2025-04-08T22:00:06Z

tests/ci/cdk/cdk/bm_framework_stack.py

 class BmFrameworkStack(Stack):
    """Define a stack used to create a CodeBuild instance on which to execute the AWS-LC benchmarking framework"""


We can actually delete this file/stack. We haven't run the benchmark stack in codebuild in years.

andrewhop · 2025-04-08T22:04:23Z

tests/ci/cdk/pipeline/ci_stage.py

+
+        self.build_options = []
+
+        # Define CodeBuild Batch job for testing code.


Can this reuse any of the existing CDK definitions? In the future if we add a new build project to our public CI we might forget to update this spot as well.

andrewhop · 2025-04-08T22:05:29Z

tests/ci/cdk/pipeline/ci_stage.py

+        # bm_framework_build_spec_file = "cdk/codebuild/bm_framework_omnibus.yaml"
+        # BmFrameworkStack(app, "aws-lc-ci-bm-framework", bm_framework_build_spec_file, env=env)


Suggested change

# bm_framework_build_spec_file = "cdk/codebuild/bm_framework_omnibus.yaml"

# BmFrameworkStack(app, "aws-lc-ci-bm-framework", bm_framework_build_spec_file, env=env)

andrewhop · 2025-04-08T22:06:02Z

tests/ci/cdk/pipeline/ci_stage.py

+
+        self.build_options = []
+
+        # Define CodeBuild Batch job for testing code.


Is this the CI that tests the code in PRs, or the CI that tests the pipeline and docker images is working?

andrewhop · 2025-04-08T22:07:27Z

tests/ci/cdk/pipeline/ci_stage.py

+                    "curl -H \"Authorization: token ${GITHUB_PAT}\" https://api.github.com/user",
+                    "git clone https://${GITHUB_PAT}@github.com/${STAGING_GITHUB_REPO_OWNER}/${STAGING_GITHUB_REPO_NAME}.git",
+                    "git remote add upstream https://github.com/aws/aws-lc.git",
+                    "git fetch upstream",
+                    "git checkout main",
+                    "git merge upstream/main",


Why are you merging aws/aws-lc into the pr branch? What happens if there is a conflict?

This is merging aws/aws-lc into our staging repo's main branch, not the pr branch. Since our staging repo is for testing only, it should never diverge from aws-lc. Conflicts should never happen. If they do, it means something is wrong with our staging repo.

andrewhop · 2025-04-08T22:10:20Z

tests/ci/cdk/pipeline/codebuild_batch_step.py

+            else {}
+        )
+
+    @jsii.member(jsii_name="produceAction")


Yeah, is there some functionality missing in the Python CDK? Have we given feedback to them about it

nhatnghiho force-pushed the ci-pipeline branch 2 times, most recently from 267feb5 to b100ece Compare March 20, 2025 22:30

nhatnghiho added 2 commits March 25, 2025 16:53

Create pipeline

45a1b91

Update cdk instructions

310e375

nhatnghiho force-pushed the ci-pipeline branch 3 times, most recently from 3f82970 to edbd376 Compare March 26, 2025 16:37

nhatnghiho marked this pull request as ready for review March 26, 2025 16:39

nhatnghiho requested a review from a team as a code owner March 26, 2025 16:39

skmcgrail self-requested a review March 26, 2025 20:34

nhatnghiho added 3 commits March 26, 2025 14:30

Add typing and cleanup

4fadbc4

Reformat code

10d511f

Fix minor bugs

7ec6289

nhatnghiho force-pushed the ci-pipeline branch from edbd376 to b93ce94 Compare March 28, 2025 20:34

nhatnghiho added 3 commits April 2, 2025 11:37

Add step to auto-sync private repo

775eeed

Fix dev pipeline deployment bug

3537e22

More cleanup

0a19c45

nhatnghiho force-pushed the ci-pipeline branch from b93ce94 to 0a19c45 Compare April 2, 2025 18:51

skmcgrail reviewed Apr 2, 2025

View reviewed changes

andrewhop reviewed Apr 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create pre-production stage for CI pipeline #2282

Create pre-production stage for CI pipeline #2282

nhatnghiho commented Mar 20, 2025 •

edited

Loading

codecov-commenter commented Mar 20, 2025 •

edited

Loading

skmcgrail Mar 26, 2025

skmcgrail Mar 26, 2025

skmcgrail Mar 26, 2025

skmcgrail Mar 26, 2025

skmcgrail Apr 2, 2025

skmcgrail Apr 2, 2025

skmcgrail Apr 2, 2025

skmcgrail Apr 2, 2025

skmcgrail Apr 2, 2025

skmcgrail Apr 2, 2025

skmcgrail Apr 2, 2025

andrewhop Apr 8, 2025

nhatnghiho Apr 9, 2025

andrewhop Apr 8, 2025

andrewhop Apr 8, 2025

andrewhop Apr 8, 2025

andrewhop Apr 8, 2025

andrewhop Apr 8, 2025

andrewhop Apr 8, 2025

nhatnghiho Apr 9, 2025

andrewhop Apr 8, 2025

		image: 620771051181.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_clang-7x-bm-framework_latest
		image: 183295444613.dkr.ecr.us-west-2.amazonaws.com/aws-lc-docker-images-linux-x86:ubuntu-20.04_clang-7x-bm-framework_latest


		assume_role

		if [[ -z "${BUILD_TYPE+x}" \|\| -z "${BUILD_TYPE}" ]]; then

	if [[ -z "${BUILD_TYPE+x}" \|\| -z "${BUILD_TYPE}" ]]; then
	if [[ -z "${BUILD_TYPE:+x}" ]]; then

		class BmFrameworkStack(Stack):
		"""Define a stack used to create a CodeBuild instance on which to execute the AWS-LC benchmarking framework"""


		self.build_options = []

		# Define CodeBuild Batch job for testing code.

		# bm_framework_build_spec_file = "cdk/codebuild/bm_framework_omnibus.yaml"
		# BmFrameworkStack(app, "aws-lc-ci-bm-framework", bm_framework_build_spec_file, env=env)

Create pre-production stage for CI pipeline #2282

Are you sure you want to change the base?

Create pre-production stage for CI pipeline #2282

Conversation

nhatnghiho commented Mar 20, 2025 • edited Loading

Issues:

Description of changes:

codecov-commenter commented Mar 20, 2025 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nhatnghiho commented Mar 20, 2025 •

edited

Loading

codecov-commenter commented Mar 20, 2025 •

edited

Loading