fix: password is empty error after latest update

[sfc-gh-twhite]

Describe the bug
User is getting “password is empty error“ in circle ci pipeline following the merge of #303.

To Reproduce
No changes were made from the previous release, the user has been using the yaml configuration.

Schemachange (please complete the following information):
99ff357

[simonB2020]

Ditto this.
I had implemented setup from this URL :
https://quickstarts.snowflake.com/guide/devops_dcm_schemachange_azure_devops/index.html?index=..%2F..index#5

Was working end Nov / early Dev
Not I just get "password is empty"

[gregory-mirsky-gfk]

I wound up pinning the SchemaChange version to 3.7.0 as a temporary fix.
echo "Step 1: Installing schemachange..."
pip install schemachange==3.7.0

[vpyao01]

I wound up pinning the SchemaChange version to 3.7.0 as a temporary fix. echo "Step 1: Installing schemachange..." pip install schemachange==3.7.0

This is a Lifesaver, thank you! We got an email from snowflake on some changes being deployed for the 7th but didn't know that it would cause something to break with the integration of both

[simonB2020]

I wound up pinning the SchemaChange version to 3.7.0 as a temporary fix. echo "Step 1: Installing schemachange..." pip install schemachange==3.7.0

Very grateful !
Thanks for sharing

[zanebclark]

The 4.0.0 release was pretty opinionated about configuration. Previously, schemachange was heavily involved in crafting the configuration parameters for the Snowflake connector. It supported the SNOWFLAKE_PASSWORD environment variable, but it didn't fully support all of the options available via the connections.toml approach. I'd like to see the python connector source configuration parameters from environment variables to achieve parity with SnowCLI. That would keep the schemachange repo out of the configuration business and return the functionality you're missing.

[dwelden]

While I understand this is unofficial software from Snowflake, it is bad form to introduce changes like this that break production deployments. Perhaps consider feature flags to help people make the transition gracefully. We are reverting back to 3.7.0 indefinitely until such time as this is handled in a more mature approach.

[zanebclark]

@dwelden , feature flags are a great idea and we will be using them to introduce new non-breaking behavior in minor updates. You might consider pinning your dependencies to avoid ingesting major releases before you're ready to consume them. Traditionally, major updates are the breaking ones. This change wouldn't have been released as a 3.x.x because of the breaking change.

[sblackstone]

What is the canonical way to import credentials that are stored in a github action secrets?

Unless I'm missing something, it seems like I'd have to build the toml file on the fly from the environment variables since there isn't pre-processing on the TOML file like there is with the YAML file.

[gregory-mirsky-gfk]

@sblackstone Correct!

Pull the secrets from the GitHUb vault into environment variables like this

jobs:
  deploy-snowflake-changes-job:
    runs-on: ubuntu-latest
    env:
      SF_ACCOUNT: ${{ secrets.SF_ACCOUNT }}
      SF_USERNAME: ${{ secrets.SF_USERNAME }}
      SF_ROLE: ${{ secrets.SF_ROLE }}
      SF_WAREHOUSE: ${{ secrets.SF_WAREHOUSE }}
      SF_DATABASE: ${{ secrets.SF_DATABASE }}
      SNOWFLAKE_PASSWORD: ${{ secrets.SF_PASSWORD }}
    steps:
...
...
...

Then create the connections.toml file on the fly with a step like this

      - name: Create and populate connections toml
        run: |
          echo "Current Directory: ${PWD}"
          echo "GITHUB WORKSPACE: ${GITHUB_WORKSPACE}"
          chmod +x connection.sh
          bash connection.sh
        working-directory: .

using a bash script called connection in the repository like this

 #!/bin/bash
# Script used in github actions to run test the schemachange functionality against the demo scenarios included in the repository.
touch ./connections.toml
echo [default] >> ./connections.toml
echo account = \"${SF_ACCOUNT}\" >> ./connections.toml
echo user = \"${SF_USERNAME}\" >> ./connections.toml
echo role = \"${SF_ROLE}\" >> ./connections.toml
echo warehouse = \"${SF_WAREHOUSE}\" >> ./connections.toml
echo database = \"${SF_DATABASE}\" >> ./connections.toml
echo password = \"${SNOWFLAKE_PASSWORD}\" >> ./connections.toml
echo "cat connections.toml"
cat ./connections.toml
echo "Current connections.toml"
ls -la ./connections.toml
echo "Changing owner and permissions to 666 for connections.toml"
sudo chmod 0600 ./connections.toml
sudo chown runner:runner ./connections.toml
ls -la ./connections.toml
pwd
#

[sblackstone]

@gregory-mirsky-gfk

This is what I did following the example in the codebase - but it feels like an anti-pattern that we have to populate a file with environment variables manually.