Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial infrastructure setup from George #29

Closed
wants to merge 31 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
3ba3961
initial infra setup, project configured with terraform
May 17, 2024
840823d
aurora postgresql provisioned
May 17, 2024
b40e677
ElasticCache provisioned
May 17, 2024
bba22b4
adding random_secret to tf
May 18, 2024
8390b3e
updating db prod config to use env vars
May 20, 2024
aaf0a4c
whoops forgot to enable the release-build make target in the cbv dire…
May 20, 2024
3fe3100
removing dev release build in favor of production
May 20, 2024
201e564
minor typo fixes to Makefile file
May 20, 2024
3ee8038
adding rails cache to gitignore
May 20, 2024
33bba8e
generated new master key
May 20, 2024
2a2d674
Remove tmp/cache/bootsnap/load-path-cache from version control, added…
May 20, 2024
045c1fd
added production key, moved secrets into rails credential file
May 20, 2024
8849238
added rails_master_key to env vars and aws, modified dockerfile to re…
May 20, 2024
c01d488
Mount /rails/tmp as temp directory
tdooner May 20, 2024
719bdd7
added RAILS_SERVE_STATIC_FILES to environment-variables.tf
May 21, 2024
52d0066
Fix permissions on /rails/tmp volume mount
tdooner May 21, 2024
33d6123
Merge remote-tracking branch 'origin/main' into george/ffs-792
tdooner May 21, 2024
8a51d1c
Attempt to run Github Actions in cbv subfolder
tdooner May 21, 2024
74c64ba
Attempt to fix broken markdown links
tdooner May 21, 2024
b1fb9b8
Fix Github actionlint issues
tdooner May 21, 2024
517ddcd
Fix hadolint warnings in cbv/Dockerfile
tdooner May 21, 2024
780f665
Add x86_64-linux platform to Gemfile
tdooner May 21, 2024
575d871
Fix checkov lints
tdooner May 21, 2024
fd83246
Disable CI/Infra/Pa11y checks
tdooner May 21, 2024
118cf21
Try fixing actions by passing `working-directory`
tdooner May 21, 2024
7e9f4bd
Remove `working-directory` from steps that call other actions
tdooner May 21, 2024
8398732
Update rspec and yarn GH action commands for cbv subdirectory
tdooner May 21, 2024
558ac9e
Fix rubocop and rspec github actions
tdooner May 21, 2024
1d223f8
Fix rubocop style error
tdooner May 22, 2024
63f140b
Clean up a bunch of accidentally-committed files
tdooner May 22, 2024
60f8265
Remove accidentally-committed .DS_Store files
tdooner May 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
6 changes: 3 additions & 3 deletions .github/actions/cleanup-gemfile/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,6 @@ runs:
- name: Remove Debug Gems
shell: bash
run: |
sed -i '/debase-ruby_core_source/d' ./Gemfile
sed -i '/debase/d' ./Gemfile
sed -i '/ruby-debug-ide/d' ./Gemfile
sed -i '/debase-ruby_core_source/d' ./cbv/Gemfile
sed -i '/debase/d' ./cbv/Gemfile
sed -i '/ruby-debug-ide/d' ./cbv/Gemfile
58 changes: 58 additions & 0 deletions .github/actions/configure-aws-credentials/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: 'Configure AWS Credentials'
description: 'Configure AWS Credentials for a given application and |
environment so that the GitHub Actions workflow can access AWS resources. |
This is a wrapper around https://github.com/aws-actions/configure-aws-credentials |
that first determines the account, role, and region based on the |
account_names_by_environment configuration in app-config'
inputs:
app_name:
description: 'Name of application folder under /infra'
required: true
environment:
description: 'Name of environment (dev, staging, prod) that AWS resources live in, or "shared" for resources that are shared across environments'
required: true
runs:
using: "composite"
steps:
- name: Get AWS account authentication details (AWS account, IAM role, AWS region)
run: |
# Get AWS account authentication details (AWS account, IAM role, AWS region)
# associated with the application environment to figure out which AWS
# account to log into, which IAM role to assume, and which AWS region to use

echo "::group::AWS account authentication details"

terraform -chdir=infra/project-config init > /dev/null
terraform -chdir=infra/project-config apply -auto-approve > /dev/null
AWS_REGION=$(terraform -chdir=infra/project-config output -raw default_region)
echo "AWS_REGION=$AWS_REGION"
GITHUB_ACTIONS_ROLE_NAME=$(terraform -chdir=infra/project-config output -raw github_actions_role_name)
echo "GITHUB_ACTIONS_ROLE_NAME=$GITHUB_ACTIONS_ROLE_NAME"

terraform -chdir=infra/${{ inputs.app_name }}/app-config init > /dev/null
terraform -chdir=infra/${{ inputs.app_name }}/app-config apply -auto-approve > /dev/null
ACCOUNT_NAME=$(terraform -chdir=infra/${{ inputs.app_name }}/app-config output -json account_names_by_environment | jq -r .${{ inputs.environment }})
echo "ACCOUNT_NAME=$ACCOUNT_NAME"

# Get the account id associated with the account name extracting the
# ACCOUNT_ID part of the tfbackend file name which looks like
# <ACCOUNT_NAME>.<ACCOUNT_ID>.s3.tfbackend.
# The cut command splits the string with period as the delimeter and
# extracts the second field.
ACCOUNT_ID=$(ls infra/accounts/$ACCOUNT_NAME.*.s3.tfbackend | cut -d. -f2)
echo "ACCOUNT_ID=$ACCOUNT_ID"

AWS_ROLE_TO_ASSUME=arn:aws:iam::$ACCOUNT_ID:role/$GITHUB_ACTIONS_ROLE_NAME
echo "AWS_ROLE_TO_ASSUME=$AWS_ROLE_TO_ASSUME"

echo "::endgroup::"

echo "Setting env vars AWS_ROLE_TO_ASSUME and AWS_REGION..."
echo "AWS_ROLE_TO_ASSUME=$AWS_ROLE_TO_ASSUME" >> "$GITHUB_ENV"
echo "AWS_REGION=$AWS_REGION" >> "$GITHUB_ENV"
shell: bash
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
with:
role-to-assume: ${{ env.AWS_ROLE_TO_ASSUME }}
aws-region: ${{ env.AWS_REGION }}
7 changes: 6 additions & 1 deletion .github/actions/setup-languages/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,22 @@ runs:

- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
working-directory: cbv

- name: Install Dependencies
shell: bash
working-directory: cbv
run: bundle install

- name: Set up node
uses: actions/setup-node@v4
with:
node-version-file: '.node-version'
node-version-file: 'cbv/.node-version'
cache-dependency-path: 'cbv/yarn.lock'
cache: 'yarn'

- name: Install yarn dependencies
shell: bash
working-directory: cbv
run: yarn install --frozen-lockfile
2 changes: 2 additions & 0 deletions .github/actions/setup-project/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,13 @@ runs:
env:
RAILS_ENV: ${{ inputs.rails_env }}
shell: bash
working-directory: cbv
run: bundle exec rake assets:precompile

- name: Set up database
env:
RAILS_ENV: ${{ inputs.rails_env }}
DATABASE_URL: ${{ inputs.database_url }}
shell: bash
working-directory: cbv
run: bundle exec rake db:schema:load
15 changes: 15 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
## Ticket

Resolves #{TICKET NUMBER OR URL}

## Changes

> What was added, updated, or removed in this PR.

## Context for reviewers

> Testing instructions, background context, more in-depth details of the implementation, and anything else you'd like to call out or ask reviewers.

## Testing

> Provide evidence that the code works as expected. Explain what was done for testing and the results of the test plan. Include screenshots, [GIF demos](https://www.cockos.com/licecap/), shell commands or output to help show the changes working as expected. ProTip: you can drag and drop or paste images into this textbox.
47 changes: 47 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# CI/CD

The CI/CD for this project uses [reusable Github Actions workflows](https://docs.github.com/en/actions/using-workflows/reusing-workflows).

## 🧪 CI

### Per app workflows

Each app should have:

- `ci-[app_name]`: must be created; should run linting and testing
- `ci-[app_name]-vulnerability-scans`: calls `vulnerability-scans`
- Based on [ci-app-vulnerability-scans](https://github.com/navapbc/template-infra/blob/main/.github/workflows/ci-app-vulnerability-scans.yml)

### App-agnostic workflows

- [`ci-docs`](./ci-docs.yml): runs markdown linting on all markdown files in the file
- Configure in [markdownlint-config.json](./markdownlint-config.json)
- [`ci-infra`](./ci-infra.yml): run infrastructure CI checks

## 🚢 CD

Each app should have:

- `cd-[app_name]`: deploys an application
- Based on [`cd-app`](https://github.com/navapbc/template-infra/blob/main/.github/workflows/cd-app.yml)

The CD workflow uses these reusable workflows:

- [`deploy`](./deploy.yml): deploys an application
- [`database-migrations`](./database-migrations.yml): runs database migrations for an application
- [`build-and-publish`](./build-and-publish.yml): builds a container image for an application and publishes it to an image repository

```mermaid
graph TD
cd-app
deploy
database-migrations
build-and-publish

cd-app-->|calls|deploy-->|calls|database-migrations-->|calls|build-and-publish
```

## ⛑️ Helper workflows

- [`check-infra-auth`](./check-infra-auth.yml): verifes that the project's Github repo is able to connect to AWS

9 changes: 6 additions & 3 deletions .github/workflows/brakeman-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,18 @@ name: Brakeman Scan
on:
push:
branches: [ main ]
paths-ignore:
- 'doc/**'
- 'README.md'
paths: ['cbv/**']
pull_request:
# The branches below must be a subset of the branches above
branches: [ main ]
paths: ['cbv/**']
schedule:
# cron format: 'minute hour dayofmonth month dayofweek'
# this will run at noon UTC each Monday (7am EST / 8am EDT)
- cron: '0 12 * * 1'
defaults:
run:
working-directory: ./cbv

jobs:
brakeman-scan:
Expand All @@ -33,5 +35,6 @@ jobs:

# Execute Brakeman CLI.
- name: Scan
working-directory: cbv
run: |
bundle exec brakeman .
51 changes: 51 additions & 0 deletions .github/workflows/build-and-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Build and publish
run-name: Build and publish ${{ inputs.app_name }}:${{ inputs.ref }}

on:
workflow_call:
inputs:
app_name:
description: "name of application folder under infra directory"
required: true
type: string
ref:
description: The branch, tag or SHA to checkout. When checking out the repository that triggered a workflow, this defaults to the reference or SHA for that event. Otherwise, use branch or tag that triggered the workflow run.
required: true
type: string
workflow_dispatch:
inputs:
app_name:
description: "name of application folder under infra directory"
required: true
type: string
ref:
description: The branch, tag or SHA to checkout. When checking out the repository that triggered a workflow, this defaults to the reference or SHA for that event. Otherwise, use branch or tag that triggered the workflow run.
required: true
type: string

jobs:
build-and-publish:
name: Build and publish
runs-on: ubuntu-latest
concurrency: ${{ github.action }}-${{ inputs.ref }}

permissions:
contents: read
id-token: write

steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.ref }}

- name: Build release
run: make APP_NAME=${{ inputs.app_name }} release-build

- name: Configure AWS credentials
uses: ./.github/actions/configure-aws-credentials
with:
app_name: ${{ inputs.app_name }}
environment: shared

- name: Publish release
run: make APP_NAME=${{ inputs.app_name }} release-publish
33 changes: 33 additions & 0 deletions .github/workflows/cd-app.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Deploy App
# Need to set a default value for when the workflow is triggered from a git push
# which bypasses the default configuration for inputs
run-name: Deploy ${{ github.ref_name }} to App ${{ inputs.environment || 'dev' }}

on:
# !! Uncomment the following lines once you've set up the dev environment and ready to turn on continuous deployment
# push:
# branches:
# - "main"
# paths:
# - "app/**"
# - "bin/**"
# - "infra/**"
workflow_dispatch:
inputs:
environment:
description: "target environment"
required: true
default: "dev"
type: choice
options:
- dev
- staging
- prod

jobs:
deploy:
name: Deploy
uses: ./.github/workflows/deploy.yml
with:
app_name: "app"
environment: ${{ inputs.environment || 'dev' }}
30 changes: 30 additions & 0 deletions .github/workflows/check-infra-auth.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Check GitHub Actions AWS Authentication

on:
workflow_dispatch:
inputs:
aws_region:
description: AWS region
default: us-east-1
required: false
role_to_assume:
description: ARN of IAM role to assume
required: true

permissions:
contents: read
id-token: write

jobs:
caller-identity:
name: Check caller identity
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
with:
aws-region: ${{ inputs.aws_region }}
role-to-assume: ${{ inputs.role_to_assume }}
- run: aws sts get-caller-identity
27 changes: 27 additions & 0 deletions .github/workflows/ci-app-vulnerability-scans.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: CI Vulnerability Scans

# TODO: Re-enable me!
on: []
# push:
# branches:
# - main
# paths:
# - cbv/**
# - .grype.yml
# - .hadolint.yaml
# - .trivyignore
# - .github/workflows/ci-app-vulnerability-scans.yml
# pull_request:
# paths:
# - cbv/**
# - .grype.yml
# - .hadolint.yaml
# - .trivyignore
# - .github/workflows/ci-app-vulnerability-scans.yml

jobs:
vulnerability-scans:
name: Vulnerability Scans
uses: ./.github/workflows/vulnerability-scans.yml
with:
app_name: "cbv"
21 changes: 21 additions & 0 deletions .github/workflows/ci-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: CI Documentation Checks

# TODO: Re-enable me!
on: []
# push:
# branches: [ main ]
# paths: ['cbv/**']
# pull_request:
# paths: ['cbv/**']

jobs:
lint-markdown:
name: Lint markdown
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
# This is the GitHub Actions-friendly port of the linter used in the Makefile.
- uses: gaurav-nelson/github-action-markdown-link-check@1.0.15
with:
use-quiet-mode: 'yes' # errors only.
config-file: '.github/workflows/markdownlint-config.json'
Loading