Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature gcp migration #3075

Merged
merged 97 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
9564549
18258 auth gcp (#2973)
pwei1018 Aug 20, 2024
b1955a5
Fixed sqlalchemy warning message.
pwei1018 Aug 21, 2024
5fd22f5
18258 auth gcp (#2973)
pwei1018 Aug 20, 2024
2a73927
Fixed sqlalchemy warning message.
pwei1018 Aug 21, 2024
6aabe94
Upgrade CI/CD.
pwei1018 Aug 28, 2024
e53f70b
Upgrade auth-web CI/CD (#2983)
pwei1018 Aug 29, 2024
0798c33
Update auth-web-cd.yml
pwei1018 Aug 29, 2024
46b8256
Feature poetry auth queue (#2995)
bolyachevets Sep 6, 2024
e94ec8c
bump python version (#2996)
bolyachevets Sep 9, 2024
aab2e35
VPC_CONNECTOR var points to shared vpc connector in common- gcp proje…
bolyachevets Sep 9, 2024
fac5536
update poetry lock after repointing to feature branch (#2999)
bolyachevets Sep 9, 2024
0d1878f
Merge branch 'main' into feature-gcp-migration
bolyachevets Sep 17, 2024
b6f7483
switched to structured logger (#3017)
bolyachevets Sep 17, 2024
e49cd57
Fix unit tests for auth-queue (#3018)
bolyachevets Sep 17, 2024
5cc7dcc
Merge branch 'main' into feature-gcp-migration
bolyachevets Sep 18, 2024
88914cd
Feature gcp migration (#3020)
bolyachevets Sep 18, 2024
530cf28
Feature gcp migration (#3022)
bolyachevets Sep 18, 2024
2b73539
fix maile cd flow (#3023)
bolyachevets Sep 18, 2024
0122131
fix workflow
bolyachevets Sep 19, 2024
be041c3
fix Dockerfiles
bolyachevets Sep 19, 2024
3cc924c
try lates poetry version with only 1 worker
bolyachevets Sep 19, 2024
b6853ff
add max worker fix to another dockerfile
bolyachevets Sep 19, 2024
0e133cd
fix vars (#3026)
bolyachevets Sep 20, 2024
4b74b77
misisng var (#3027)
bolyachevets Sep 20, 2024
edf4f99
update version number for release 2.10.5 (#2985)
avni-work Aug 29, 2024
b387965
fix var (#3028)
bolyachevets Sep 20, 2024
bf514fb
Fixed history table, token and linting issues.
pwei1018 Sep 20, 2024
22ecef2
Remove version tables from script.
pwei1018 Sep 20, 2024
c070212
Remove table alteration from script.
pwei1018 Sep 20, 2024
458939f
Fixed history table, token and linting issues. (#3029)
pwei1018 Sep 20, 2024
a27d537
Switch back to psycopg2 from pg8000.
pwei1018 Sep 20, 2024
f431c6b
Switch back to psycopg2 from pg8000. (#3030)
pwei1018 Sep 20, 2024
b03e108
Added missing db name.
pwei1018 Sep 20, 2024
02f922c
Add psycopg2 library.
pwei1018 Sep 20, 2024
901c64a
Add psycopg2 library. (#3031)
pwei1018 Sep 20, 2024
d19e944
Update CD flow.
pwei1018 Sep 20, 2024
35a1574
Back to pg8000 and add type of id to the model functions.
pwei1018 Sep 20, 2024
b2c7fc9
Fixed the host string.
pwei1018 Sep 20, 2024
6178501
Back to pg8000 and add type of id to the model functions. (#3032)
pwei1018 Sep 20, 2024
3ca76e6
Force convert string to int
pwei1018 Sep 20, 2024
5351494
Force convert string to int (#3033)
pwei1018 Sep 20, 2024
696ad16
Force conver string to int part 2.
pwei1018 Sep 20, 2024
3364f24
Force conver string to int part 2. (#3034)
pwei1018 Sep 20, 2024
9e21ae8
Update affiliation_invitation.py
pwei1018 Sep 20, 2024
b4a2f52
Update affiliation_invitation.py
pwei1018 Sep 20, 2024
e37160c
Update affiliation_invitation.py
pwei1018 Sep 21, 2024
9879624
update version number for release 2.10.5 (#2985)
avni-work Aug 29, 2024
ace24bd
Fixed history table, token and linting issues.
pwei1018 Sep 20, 2024
aac5838
Remove version tables from script.
pwei1018 Sep 20, 2024
9692c5c
Remove table alteration from script.
pwei1018 Sep 20, 2024
cd449d2
Switch back to psycopg2 from pg8000.
pwei1018 Sep 20, 2024
3664218
Force conver string to int.
pwei1018 Sep 20, 2024
e3315f0
Merge branch 'feature-gcp-migration' into origin/feature-gcp-migration
pwei1018 Sep 21, 2024
879f024
Fixed CD.
pwei1018 Sep 21, 2024
78b3fc0
Merge branch 'origin/feature-gcp-migration' of https://github.com/pwe…
pwei1018 Sep 21, 2024
80f8a4a
Fixed linting.
pwei1018 Sep 21, 2024
2fb4a02
Fixed pg8000 integer field issue. (#3035)
pwei1018 Sep 21, 2024
023626b
fix data: null response when {} is returned (#3037)
bolyachevets Sep 23, 2024
091b305
another type error (#3038)
bolyachevets Sep 24, 2024
d9ed03a
Feature gcp migration (#3039)
bolyachevets Sep 24, 2024
6d5f1a1
Fix poetry locks (#3042)
bolyachevets Sep 24, 2024
2fc81a4
Merge branch 'main' into feature-gcp-migration
bolyachevets Sep 24, 2024
1f79323
Fix circular dependency error from last merge (#3045)
bolyachevets Sep 24, 2024
bfbd4c2
add stack trace to db error / path to http error logs (#3046)
bolyachevets Sep 25, 2024
2eacef9
pg8000 type error / add stacktrace for std handler (#3048)
bolyachevets Sep 26, 2024
3c661b5
add more details for http exception logging (#3049)
bolyachevets Sep 26, 2024
f50c87c
fix keycloak base_url
bolyachevets Sep 27, 2024
90ffa30
fix keycloak params (#3051)
pwei1018 Sep 27, 2024
c01c15e
Fixed org products endpoint 404 issue.
pwei1018 Sep 27, 2024
f3882ae
Fixed org products endpoint 404 issue. (#3052)
pwei1018 Sep 27, 2024
62162fe
Fixed pg8000 integer issue.
pwei1018 Sep 27, 2024
dd6259a
Fixed typo.
pwei1018 Sep 27, 2024
7d1cfbc
Fixed pg8000 integer issue. (#3053)
pwei1018 Sep 27, 2024
ae03290
Fixed get products issue.
pwei1018 Sep 28, 2024
c571a70
Fixed get products issue. (#3054)
pwei1018 Sep 28, 2024
38d96f4
fix pipe name (#3057)
bolyachevets Oct 1, 2024
553021c
Removed id foreign keys in history tables. (#3058)
pwei1018 Oct 1, 2024
d06451a
add resource params (#3059)
bolyachevets Oct 2, 2024
60ccbe4
fix resp (#3060)
bolyachevets Oct 2, 2024
67b5966
undo (#3061)
bolyachevets Oct 2, 2024
6b87ca5
fix typo (#3062)
bolyachevets Oct 2, 2024
b64f247
test custom camel case function (#3063)
bolyachevets Oct 3, 2024
3e0ab60
use modified camelcase function (#3064)
bolyachevets Oct 3, 2024
217f713
Fixed org schema mailing address issue and auth error logging.
pwei1018 Oct 1, 2024
be44011
Fixed migration script issue.
pwei1018 Oct 3, 2024
181d257
Remove all fks from history tables.
pwei1018 Oct 3, 2024
450c8ee
Fixed org schema mailing address issue and auth error logging. (#3065)
pwei1018 Oct 3, 2024
18242f7
add better logging to services (#3067)
bolyachevets Oct 3, 2024
8f4ac80
Remove migration out of app start and fix linting.
pwei1018 Oct 3, 2024
289c7db
Remove migration out of app start and fix linting. (#3068)
pwei1018 Oct 4, 2024
17b9155
Fxied migration script issue.
pwei1018 Oct 4, 2024
38ee18e
Fixed linting.
pwei1018 Oct 4, 2024
e685f85
Fxied migration script issue. (#3071)
pwei1018 Oct 4, 2024
d873fd9
fix report url (#3073)
bolyachevets Oct 7, 2024
636edcd
Merge branch 'main' into feature-gcp-migration
seeker25 Oct 8, 2024
8fb8189
fix linting
seeker25 Oct 8, 2024
c2359e8
missing system role + other merge conflict issues
seeker25 Oct 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
119 changes: 18 additions & 101 deletions .github/workflows/account-mailer-cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ on:
push:
branches:
- main
- feature*
paths:
- "queue_services/account-mailer/**"
- "auth-api/src/auth_api/models/**"
Expand All @@ -12,106 +13,22 @@ on:
- "auth-api/src/auth_api/utils/enums.py"
workflow_dispatch:
inputs:
environment:
description: "Environment (dev/test/prod)"
target:
description: "Deploy To"
required: true
default: "dev"

defaults:
run:
shell: bash
working-directory: ./queue_services/account-mailer

env:
APP_NAME: "account-mailer"
TAG_NAME: "dev"

type: choice
options:
- dev
- test
- sandbox
- prod
jobs:
account-mailer-cd-by-push:
runs-on: ubuntu-20.04

if: github.event_name == 'push' && github.repository == 'bcgov/sbc-auth'
environment:
name: "dev"

steps:
- uses: actions/checkout@v4

- name: Login Openshift
shell: bash
run: |
oc login --server=${{secrets.OPENSHIFT4_LOGIN_REGISTRY}} --token=${{secrets.OPENSHIFT4_SA_TOKEN}}

- name: CD Flow
shell: bash
env:
OPS_REPOSITORY: ${{ secrets.OPS_REPOSITORY }}
OPENSHIFT_DOCKER_REGISTRY: ${{ secrets.OPENSHIFT4_DOCKER_REGISTRY }}
OPENSHIFT_SA_NAME: ${{ secrets.OPENSHIFT4_SA_NAME }}
OPENSHIFT_SA_TOKEN: ${{ secrets.OPENSHIFT4_SA_TOKEN }}
OPENSHIFT_REPOSITORY: ${{ secrets.OPENSHIFT4_REPOSITORY }}
TAG_NAME: ${{ env.TAG_NAME }}
run: |
make cd

- name: Watch new rollout (trigger by image change in Openshift)
shell: bash
run: |
oc rollout status dc/${{ env.APP_NAME }}-${{ env.TAG_NAME }} -n ${{ secrets.OPENSHIFT4_REPOSITORY }}-${{ env.TAG_NAME }} -w

- name: Rocket.Chat Notification
uses: RocketChat/Rocket.Chat.GitHub.Action.Notification@master
if: failure()
with:
type: ${{ job.status }}
job_name: "*Account Mailer Built and Deployed to ${{env.TAG_NAME}}*"
channel: "#registries-bot"
url: ${{ secrets.ROCKETCHAT_WEBHOOK }}
commit: true
token: ${{ secrets.GITHUB_TOKEN }}

account-mailer-cd-by-dispatch:
runs-on: ubuntu-20.04

if: github.event_name == 'workflow_dispatch' && github.repository == 'bcgov/sbc-auth'
environment:
name: "${{ github.event.inputs.environment }}"

steps:
- uses: actions/checkout@v4
- name: Set env by input
run: |
echo "TAG_NAME=${{ github.event.inputs.environment }}" >> $GITHUB_ENV

- name: Login Openshift
shell: bash
run: |
oc login --server=${{secrets.OPENSHIFT4_LOGIN_REGISTRY}} --token=${{secrets.OPENSHIFT4_SA_TOKEN}}

- name: CD Flow
shell: bash
env:
OPS_REPOSITORY: ${{ secrets.OPS_REPOSITORY }}
OPENSHIFT_DOCKER_REGISTRY: ${{ secrets.OPENSHIFT4_DOCKER_REGISTRY }}
OPENSHIFT_SA_NAME: ${{ secrets.OPENSHIFT4_SA_NAME }}
OPENSHIFT_SA_TOKEN: ${{ secrets.OPENSHIFT4_SA_TOKEN }}
OPENSHIFT_REPOSITORY: ${{ secrets.OPENSHIFT4_REPOSITORY }}
TAG_NAME: ${{ env.TAG_NAME }}
run: |
make cd

- name: Watch new rollout (trigger by image change in Openshift)
shell: bash
run: |
oc rollout status dc/${{ env.APP_NAME }}-${{ env.TAG_NAME }} -n ${{ secrets.OPENSHIFT4_REPOSITORY }}-${{ env.TAG_NAME }} -w

- name: Rocket.Chat Notification
uses: RocketChat/Rocket.Chat.GitHub.Action.Notification@master
if: failure()
with:
type: ${{ job.status }}
job_name: "*Account Mailer Built and Deployed to ${{env.TAG_NAME}}*"
channel: "#registries-bot"
url: ${{ secrets.ROCKETCHAT_WEBHOOK }}
commit: true
token: ${{ secrets.GITHUB_TOKEN }}
account-mailer-cd:
uses: bcgov/bcregistry-sre/.github/workflows/backend-cd.yaml@main
with:
target: ${{ inputs.target }}
app_name: "account-mailer"
working_directory: "./queue_services/account-mailer"
secrets:
WORKLOAD_IDENTIFY_POOLS_PROVIDER: ${{ secrets.WORKLOAD_IDENTIFY_POOLS_PROVIDER }}
GCP_SERVICE_ACCOUNT: ${{ secrets.GCP_SERVICE_ACCOUNT }}
8 changes: 6 additions & 2 deletions .github/workflows/account-mailer-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ on:
- "auth-api/src/auth_api/services/gcp_queue/*"
- "auth-api/src/auth_api/utils/account_mailer.py"
- "auth-api/src/auth_api/utils/enums.py"
- "build-deps/**"

defaults:
run:
Expand All @@ -32,12 +33,12 @@ jobs:

strategy:
matrix:
python-version: [3.8]
python-version: [3.12]

steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
Expand Down Expand Up @@ -69,6 +70,9 @@ jobs:
BCOL_ADMIN_EMAIL: "test@test.com"

runs-on: ubuntu-20.04
strategy:
matrix:
python-version: [3.12]

services:
postgres:
Expand Down
118 changes: 18 additions & 100 deletions .github/workflows/auth-api-cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,110 +4,28 @@ on:
push:
branches:
- main
- feature*
paths:
- "auth-api/**"
workflow_dispatch:
inputs:
environment:
description: "Environment (dev/test/prod)"
target:
description: "Deploy To"
required: true
default: "dev"

defaults:
run:
shell: bash
working-directory: ./auth-api

env:
APP_NAME: "auth-api"
TAG_NAME: "dev"
type: choice
options:
- dev
- test
- sandbox
- prod

jobs:
auth-api-cd-by-push:
runs-on: ubuntu-20.04

if: github.event_name == 'push' && github.repository == 'bcgov/sbc-auth'
environment:
name: "dev"

steps:
- uses: actions/checkout@v4

- name: Login Openshift
shell: bash
run: |
oc login --server=${{secrets.OPENSHIFT4_LOGIN_REGISTRY}} --token=${{secrets.OPENSHIFT4_SA_TOKEN}}

- name: CD Flow
shell: bash
env:
OPS_REPOSITORY: ${{ secrets.OPS_REPOSITORY }}
OPENSHIFT_DOCKER_REGISTRY: ${{ secrets.OPENSHIFT4_DOCKER_REGISTRY }}
OPENSHIFT_SA_NAME: ${{ secrets.OPENSHIFT4_SA_NAME }}
OPENSHIFT_SA_TOKEN: ${{ secrets.OPENSHIFT4_SA_TOKEN }}
OPENSHIFT_REPOSITORY: ${{ secrets.OPENSHIFT4_REPOSITORY }}
TAG_NAME: ${{ env.TAG_NAME }}
run: |
make cd

- name: Watch new rollout (trigger by image change in Openshift)
shell: bash
run: |
oc rollout status dc/${{ env.APP_NAME }}-${{ env.TAG_NAME }} -n ${{ secrets.OPENSHIFT4_REPOSITORY }}-${{ env.TAG_NAME }} -w

- name: Rocket.Chat Notification
uses: RocketChat/Rocket.Chat.GitHub.Action.Notification@master
if: failure()
with:
type: ${{ job.status }}
job_name: "*Auth API Built and Deployed to ${{env.TAG_NAME}}*"
channel: "#registries-bot"
url: ${{ secrets.ROCKETCHAT_WEBHOOK }}
commit: true
token: ${{ secrets.GITHUB_TOKEN }}

auth-api-cd-by-dispatch:
runs-on: ubuntu-20.04

if: github.event_name == 'workflow_dispatch' && github.repository == 'bcgov/sbc-auth'
environment:
name: "${{ github.event.inputs.environment }}"

steps:
- uses: actions/checkout@v4
- name: Set env by input
run: |
echo "TAG_NAME=${{ github.event.inputs.environment }}" >> $GITHUB_ENV

- name: Login Openshift
shell: bash
run: |
oc login --server=${{secrets.OPENSHIFT4_LOGIN_REGISTRY}} --token=${{secrets.OPENSHIFT4_SA_TOKEN}}

- name: CD Flow
shell: bash
env:
OPS_REPOSITORY: ${{ secrets.OPS_REPOSITORY }}
OPENSHIFT_DOCKER_REGISTRY: ${{ secrets.OPENSHIFT4_DOCKER_REGISTRY }}
OPENSHIFT_SA_NAME: ${{ secrets.OPENSHIFT4_SA_NAME }}
OPENSHIFT_SA_TOKEN: ${{ secrets.OPENSHIFT4_SA_TOKEN }}
OPENSHIFT_REPOSITORY: ${{ secrets.OPENSHIFT4_REPOSITORY }}
TAG_NAME: ${{ env.TAG_NAME }}
run: |
make cd

- name: Watch new rollout (trigger by image change in Openshift)
shell: bash
run: |
oc rollout status dc/${{ env.APP_NAME }}-${{ env.TAG_NAME }} -n ${{ secrets.OPENSHIFT4_REPOSITORY }}-${{ env.TAG_NAME }} -w

- name: Rocket.Chat Notification
uses: RocketChat/Rocket.Chat.GitHub.Action.Notification@master
if: failure()
with:
type: ${{ job.status }}
job_name: "*Auth API Built and Deployed to ${{env.TAG_NAME}}*"
channel: "#registries-bot"
url: ${{ secrets.ROCKETCHAT_WEBHOOK }}
commit: true
token: ${{ secrets.GITHUB_TOKEN }}
auth-api-cd:
uses: bcgov/bcregistry-sre/.github/workflows/backend-cd.yaml@main
with:
target: ${{ inputs.target }}
app_name: "auth-api"
working_directory: "./auth-api"
secrets:
WORKLOAD_IDENTIFY_POOLS_PROVIDER: ${{ secrets.WORKLOAD_IDENTIFY_POOLS_PROVIDER }}
GCP_SERVICE_ACCOUNT: ${{ secrets.GCP_SERVICE_ACCOUNT }}
Loading
Loading