Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Fix ENV VAR kv parsing to handle JSON values #370

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

bHacklv
Copy link

@bHacklv bHacklv commented Nov 6, 2024

What

JOB_KUBE_ANNOTATIONS environment variable can be used to define one or more Job pod annotations. The valid value can be a JSON value (see example below). Current logic to process the key:value pairs is not allowing to pass the JSON value.

Before fix:

Image

Example of a valid istio k8s annotation that is incorrectly parsed without a fix:

proxy.istio.io/config={ 
"holdApplicationUntilProxyStarts": true, 
"proxyMetadata": { 
  "EXIT_ON_ZERO_ACTIVE_CONNECTIONS": "true" 
  } 
}

How

  1. Parse ENV VAR value with Regex Pattern that checks for JSON values
  2. Fallback to Regex Pattern that extracts k:v as it was done before with Splitter stream.

Recommended reading order

  1. airbyte-commons-with-dependencies/src/main/java/io/airbyte/commons/workers/config/EnvUtils.java
  2. airbyte-commons-with-dependencies/src/main/java/io/airbyte/commons/workers/config/WorkerConfigsProvider.java
  3. airbyte-commons-with-dependencies/src/test/java/io/airbyte/commons/workers/config/EnvUtilsTest.java

Can this PR be safely reverted and rolled back?

  • YES 💚
  • NO ❌

@CLAassistant
Copy link

CLAassistant commented Nov 6, 2024

CLA assistant check
All committers have signed the CLA.

@bHacklv bHacklv changed the title fix: ENV VAR kv parsing to handle json values fix: ENV VAR kv parsing to handle JSON values Nov 6, 2024
@bHacklv bHacklv force-pushed the bHacklv/kv-env-parser-fix branch 5 times, most recently from eab0669 to aeb73b4 Compare November 13, 2024 06:32
@bHacklv bHacklv force-pushed the bHacklv/kv-env-parser-fix branch 2 times, most recently from d1cedc8 to 815223f Compare November 20, 2024 20:56
@bHacklv bHacklv force-pushed the bHacklv/kv-env-parser-fix branch from 9a8f874 to 8decfab Compare November 21, 2024 10:31
@bHacklv bHacklv force-pushed the bHacklv/kv-env-parser-fix branch from 8decfab to 8931902 Compare December 7, 2024 00:52
@bHacklv bHacklv force-pushed the bHacklv/kv-env-parser-fix branch from 8931902 to aea7b63 Compare January 28, 2025 10:12
@bHacklv bHacklv changed the title fix: ENV VAR kv parsing to handle JSON values fix: ENV VAR kv parsing to handle JSON values 🐛 Jan 28, 2025
@bHacklv bHacklv changed the title fix: ENV VAR kv parsing to handle JSON values 🐛 🐛 Fix ENV VAR kv parsing to handle JSON values Jan 28, 2025
@bHacklv bHacklv force-pushed the bHacklv/kv-env-parser-fix branch from aea7b63 to 8510a86 Compare January 28, 2025 15:25
@colesnodgrass
Copy link
Member

This PR will need to be updated against the main branch as it has dependencies which no longer exist within the platform (i.e. lombok).

- JOB_KUBE_ANNOTATIONS can have JSON as value
- This fixes the parsin of such values
@bHacklv bHacklv force-pushed the bHacklv/kv-env-parser-fix branch from 8510a86 to 8d257be Compare January 30, 2025 23:27
@bHacklv
Copy link
Author

bHacklv commented Jan 30, 2025

This PR will need to be updated against the main branch as it has dependencies which no longer exist within the platform (i.e. lombok).

Sure, I will do it on Monday, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants