-
Notifications
You must be signed in to change notification settings - Fork 9
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
73eba98
commit 9df2c51
Showing
173 changed files
with
20,976 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
* @agnessnowplow |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
--- | ||
name: Bug report | ||
about: Report a bug or an issue you've found with this package | ||
title: '' | ||
labels: type:bug | ||
assignees: '' | ||
|
||
--- | ||
|
||
## Describe the bug | ||
<!--- | ||
A clear and concise description of what the bug is. | ||
---> | ||
|
||
## Steps to reproduce | ||
<!--- | ||
In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example model code, etc is all very helpful here. | ||
---> | ||
|
||
## Expected results | ||
<!--- | ||
A clear and concise description of what you expected to happen. | ||
---> | ||
|
||
## Actual results | ||
<!--- | ||
A clear and concise description of what actually happens. | ||
---> | ||
|
||
## Screenshots and log output | ||
<!--- | ||
If applicable, add screenshots or log output to help explain your problem. | ||
---> | ||
|
||
## System information | ||
**The contents of your `packages.yml` file:** | ||
```yml | ||
# contents goes here | ||
``` | ||
|
||
**Which database are you using dbt with?** | ||
- [ ] postgres | ||
- [ ] redshift | ||
- [ ] bigquery | ||
- [ ] snowflake | ||
- [ ] databricks | ||
- [ ] other (specify: ____________) | ||
|
||
**The output of `dbt --version`:** | ||
``` | ||
<output goes here> | ||
``` | ||
|
||
**The operating system you're using:** | ||
|
||
|
||
**The output of `python --version`:** | ||
|
||
|
||
## Additional context | ||
<!--- | ||
Add any other context about the problem here. For example, if you think you know which line of code is causing the issue. | ||
---> | ||
|
||
## Are you interested in contributing towards the fix? | ||
<!--- | ||
Let us know if you want to contribute towards the fix, and whether you would need a hand getting started | ||
---> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
--- | ||
name: Feature request | ||
about: Suggest an idea for this package | ||
title: '' | ||
labels: type:enhancement | ||
assignees: '' | ||
|
||
--- | ||
|
||
## Is your feature request related to a problem? Please describe. | ||
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] --> | ||
|
||
## Describe the solution you'd like | ||
<!-- A clear and concise description of what you want to happen. --> | ||
|
||
## Describe alternatives you've considered | ||
<!-- A clear and concise description of any alternative solutions or features you've considered. --> | ||
|
||
## Additional context | ||
<!-- Add any other context or screenshots about the feature request here. --> | ||
|
||
## Are you interested in contributing towards this feature? | ||
<!--- | ||
Let us know if you want to contribute towards the feature, and whether you would need a hand getting started | ||
---> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
--- | ||
name: Question | ||
about: A more general question about the package. Consider using discourse if more applicable. | ||
title: '' | ||
labels: type:question | ||
assignees: '' | ||
|
||
--- | ||
|
||
<!-- Please consider if your question would be better placed on the Snowplow Discourse (https://discourse.snowplow.io/) for more open conversations --> | ||
|
||
### Question | ||
<!-- What is your question --> | ||
|
||
### Additional context | ||
<!-- Is this feature database-specific? Which database(s) is/are relevant? Please include any other relevant context here. --> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
- name: "category:breaking_change" | ||
color: '1e1037' | ||
description: "A breaking change will be introduced if this issue is completed." | ||
- name: "category:documentation" | ||
color: '3849b8' | ||
description: "About the documentation." | ||
- name: "cla:no" | ||
color: '313131' | ||
description: "[Auto generated] Snowplow Contributor License Agreement has not been signed." | ||
- name: "cla:yes" | ||
color: 'c1c1c1' | ||
description: "[Auto generated] Snowplow Contributor License Agreement has been signed." | ||
- name: "good first issue" | ||
color: '69f6ff' | ||
description: "Good issue for a first time contributor." | ||
- name: "priority:high" | ||
color: 'f1ff1d' | ||
description: "To fix as soon as possible." | ||
- name: "priority:low" | ||
color: 'ffd0d4' | ||
description: "Not on the roadmap." | ||
- name: "priority:medium" | ||
color: 'ff6372' | ||
description: "On the roadmap." | ||
- name: "status:blocked" | ||
color: '042e31' | ||
description: "Maintainers can't work on this right now." | ||
- name: "status:completed" | ||
color: '0ea417' | ||
description: "Completed - but might not be released yet." | ||
- name: "status:duplicate" | ||
color: 'b6e1e3' | ||
description: "Duplicates another issue." | ||
- name: "status:has_pr" | ||
color: '0ea462' | ||
description: "A PR exists for this issue." | ||
- name: "status:in_progress" | ||
color: '0e9ba4' | ||
description: "Maintainers are working on this." | ||
- name: "status:needs_triage" | ||
color: 'b6e3cf' | ||
description: "Needs maintainer triage." | ||
- name: "status:wont_fix" | ||
color: '04311d' | ||
description: "Can't or won't be fixed." | ||
- name: "status:info_needed" | ||
color: '56B9BF' | ||
description: "More information is needed from filer for issue to be actionable" | ||
- name: "status:do_not_merge" | ||
color: 'D93F0B' | ||
description: "Flag to denote a Issue or PR which should not yet be merged (usually pending a release)" | ||
- name: "status:help_wanted" | ||
color: '69f6ff' | ||
description: "Help is wanted to help solve this issue." | ||
- name: "type:admin" | ||
color: 'fdffd2' | ||
description: 'About code or repo management.' | ||
- name: "type:bug" | ||
color: '4c381f' | ||
description: 'Bugs or weaknesses. The issue has to contain steps to reproduce.' | ||
- name: "type:enhancement" | ||
color: 'ffbd69' | ||
description: 'New features or improvements to existing features.' | ||
- name: "type:question" | ||
color: 'ffebd2' | ||
description: "Questions to the maintainers." | ||
- name: "category:models" | ||
color: '6738b8' | ||
description: "Related to the models in the package." | ||
- name: "category:macros" | ||
color: 'f1ff1d' | ||
description: "Related to the macros in the package." | ||
- name: "category:tests" | ||
color: 'a738b8' | ||
description: "Related to the tests in the package." | ||
- name: "category:other" | ||
color: '6372ff' | ||
description: "Related to other parts of the package or repo itself." | ||
#- name: "category:4" | ||
# color: '9e62dd' | ||
# description: "Desc" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
<!-- | ||
If this is your first time contributing you will be asked to sign the Individual Contributor License Agreement. | ||
If you would prefer to read this in advance of submitting your PR you can find it here https://docs.google.com/forms/d/e/1FAIpQLSd89YTDQ1XpTZbj3LpOkquV_h1Y8k9ay3iFbJsZsJrz18I23Q/viewform | ||
--> | ||
|
||
## Description & motivation | ||
<!-- | ||
Describe your changes, and why you're making them. | ||
--> | ||
|
||
## Checklist | ||
- [ ] I have verified that these changes work locally | ||
- [ ] I have updated the README.md (if applicable) | ||
- [ ] I have added tests & descriptions to my models (and macros if applicable) | ||
- [ ] I have raised a [documentation](https://github.com/snowplow/documentation) PR if applicable (Link here if required) | ||
- [ ] Is your change a breaking change? | ||
|
||
<!-- | ||
## Release Only Checklist | ||
- [ ] I have updated the version number in all relevant places | ||
- [ ] I have updated the CHANGELOG.md | ||
--> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
name: No Response | ||
|
||
# Both `issue_comment` and `scheduled` event types are required for this Action | ||
# to work properly. | ||
on: | ||
issue_comment: | ||
types: [created] | ||
schedule: | ||
# Schedule for 08:00 every day | ||
- cron: '0 8 * * *' | ||
|
||
jobs: | ||
noResponse: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: lee-dohm/no-response@v0.5.0 | ||
with: | ||
token: ${{ github.token }} | ||
daysUntilClose: 14 | ||
responseRequiredLabel: "status:info_needed" | ||
closeComment: > | ||
This issue has been automatically closed because there has been no response | ||
to our request for more information from the original author within 14 days. With only what | ||
is currently in the issue, we don't have enough information | ||
to take action. Please comment with this information if you have it to reopen the issue so | ||
that we can investigate further. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,138 @@ | ||
name: pr_tests | ||
|
||
on: | ||
pull_request: | ||
|
||
concurrency: dbt_integration_tests | ||
|
||
env: | ||
# Set profiles.yml directory | ||
DBT_PROFILES_DIR: ./ci | ||
|
||
# Redshift Connection | ||
REDSHIFT_TEST_HOST: ${{ secrets.REDSHIFT_TEST_HOST }} | ||
REDSHIFT_TEST_USER: ${{ secrets.REDSHIFT_TEST_USER }} | ||
REDSHIFT_TEST_PASS: ${{ secrets.REDSHIFT_TEST_PASS }} | ||
REDSHIFT_TEST_DBNAME: ${{ secrets.REDSHIFT_TEST_DBNAME }} | ||
REDSHIFT_TEST_PORT: ${{ secrets.REDSHIFT_TEST_PORT }} | ||
|
||
# BigQuery Connection | ||
BIGQUERY_TEST_DATABASE: ${{ secrets.BIGQUERY_TEST_DATABASE }} | ||
BIGQUERY_LOCATION: ${{ secrets.BIGQUERY_LOCATION }} | ||
BIGQUERY_SERVICE_TYPE: ${{ secrets.BIGQUERY_SERVICE_TYPE }} | ||
BIGQUERY_SERVICE_PROJECT_ID: ${{ secrets.BIGQUERY_SERVICE_PROJECT_ID }} | ||
BIGQUERY_SERVICE_PRIVATE_KEY_ID: ${{ secrets.BIGQUERY_SERVICE_PRIVATE_KEY_ID }} | ||
BIGQUERY_SERVICE_PRIVATE_KEY: ${{ secrets.BIGQUERY_SERVICE_PRIVATE_KEY }} | ||
BIGQUERY_SERVICE_CLIENT_EMAIL: ${{ secrets.BIGQUERY_SERVICE_CLIENT_EMAIL }} | ||
BIGQUERY_SERVICE_CLIENT_ID: ${{ secrets.BIGQUERY_SERVICE_CLIENT_ID }} | ||
BIGQUERY_SERVICE_AUTH_URI: ${{ secrets.BIGQUERY_SERVICE_AUTH_URI }} | ||
BIGQUERY_SERVICE_TOKEN_URI: ${{ secrets.BIGQUERY_SERVICE_TOKEN_URI }} | ||
BIGQUERY_SERVICE_AUTH_PROVIDER_X509_CERT_URL: ${{ secrets.BIGQUERY_SERVICE_AUTH_PROVIDER_X509_CERT_URL }} | ||
BIGQUERY_SERVICE_CLIENT_X509_CERT_URL: ${{ secrets.BIGQUERY_SERVICE_CLIENT_X509_CERT_URL }} | ||
|
||
# Snowflake Connection | ||
SNOWFLAKE_TEST_ACCOUNT: ${{ secrets.SNOWFLAKE_TEST_ACCOUNT }} | ||
SNOWFLAKE_TEST_USER: ${{ secrets.SNOWFLAKE_TEST_USER }} | ||
SNOWFLAKE_TEST_PASSWORD: ${{ secrets.SNOWFLAKE_TEST_PASSWORD }} | ||
SNOWFLAKE_TEST_ROLE: ${{ secrets.SNOWFLAKE_TEST_ROLE }} | ||
SNOWFLAKE_TEST_DATABASE: ${{ secrets.SNOWFLAKE_TEST_DATABASE }} | ||
SNOWFLAKE_TEST_WAREHOUSE: ${{ secrets.SNOWFLAKE_TEST_WAREHOUSE }} | ||
|
||
# Postgres Connection | ||
POSTGRES_TEST_HOST: ${{ secrets.POSTGRES_TEST_HOST }} | ||
POSTGRES_TEST_USER: ${{ secrets.POSTGRES_TEST_USER }} | ||
POSTGRES_TEST_PASS: ${{ secrets.POSTGRES_TEST_PASS }} | ||
POSTGRES_TEST_PORT: ${{ secrets.POSTGRES_TEST_PORT }} | ||
POSTGRES_TEST_DBNAME: ${{ secrets.POSTGRES_TEST_DBNAME }} | ||
|
||
# Databricks Connection | ||
DATABRICKS_TEST_HOST: ${{ secrets.DATABRICKS_TEST_HOST }} | ||
DATABRICKS_TEST_HTTP_PATH: ${{ secrets.DATABRICKS_TEST_HTTP_PATH }} | ||
DATABRICKS_TEST_TOKEN: ${{ secrets.DATABRICKS_TEST_TOKEN }} | ||
DATABRICKS_TEST_ENDPOINT: ${{ secrets.DATABRICKS_TEST_ENDPOINT }} | ||
|
||
jobs: | ||
pr_tests: | ||
name: pr_tests | ||
runs-on: ubuntu-latest | ||
defaults: | ||
run: | ||
# Run tests from integration_tests sub dir | ||
working-directory: ./integration_tests | ||
strategy: | ||
matrix: | ||
dbt_version: ["1.*"] | ||
warehouse: ["postgres", "bigquery", "snowflake", "databricks"] # TODO: Add RS self-hosted runner | ||
|
||
services: | ||
postgres: | ||
image: postgres:latest | ||
env: | ||
POSTGRES_DB: ${{ secrets.POSTGRES_TEST_DBNAME }} | ||
POSTGRES_USER: ${{ secrets.POSTGRES_TEST_USER }} | ||
POSTGRES_PASSWORD: ${{ secrets.POSTGRES_TEST_PASS }} | ||
# Set health checks to wait until postgres has started | ||
options: >- | ||
--health-cmd pg_isready | ||
--health-interval 10s | ||
--health-timeout 5s | ||
--health-retries 5 | ||
ports: | ||
# Maps tcp port 5432 on service container to the host | ||
- 5432:5432 | ||
|
||
steps: | ||
- name: Check out | ||
uses: actions/checkout@v3 | ||
|
||
# Remove '*' and replace '.' with '_' in DBT_VERSION & set as SCHEMA_SUFFIX. | ||
# SCHEMA_SUFFIX allows us to run multiple versions of dbt in parallel without overwriting the output tables | ||
- name: Set SCHEMA_SUFFIX env | ||
run: echo "SCHEMA_SUFFIX=$(echo ${DBT_VERSION%.*} | tr . _)" >> $GITHUB_ENV | ||
env: | ||
DBT_VERSION: ${{ matrix.dbt_version }} | ||
|
||
- name: Set DEFAULT_TARGET env | ||
run: | | ||
echo "DEFAULT_TARGET=${{ matrix.warehouse }}" >> $GITHUB_ENV | ||
- name: Python setup | ||
uses: actions/setup-python@v4 | ||
with: | ||
python-version: "3.8.x" | ||
|
||
- name: Pip cache | ||
uses: actions/cache@v3 | ||
with: | ||
path: ~/.cache/pip | ||
key: ${{ runner.os }}-pip-${{ matrix.dbt_version }}-${{ matrix.warehouse }} | ||
restore-keys: | | ||
${{ runner.os }}-pip-${{ matrix.dbt_version }}-${{ matrix.warehouse }} | ||
# Install latest patch version. Upgrade if cache contains old patch version. | ||
- name: Install dependencies | ||
run: | | ||
pip install --upgrade pip wheel setuptools | ||
pip install -Iv dbt-${{ matrix.warehouse }}==${{ matrix.dbt_version }} --upgrade | ||
dbt deps | ||
if: ${{matrix.warehouse != 'spark'}} | ||
|
||
|
||
- name: Install spark dependencies | ||
run: | | ||
pip install --upgrade pip wheel setuptools | ||
pip install -Iv "dbt-${{ matrix.warehouse }}[ODBC]"==${{ matrix.dbt_version }} --upgrade | ||
dbt deps | ||
if: ${{matrix.warehouse == 'spark'}} | ||
|
||
- name: "Pre-test: Drop ci schemas" | ||
run: | | ||
dbt run-operation post_ci_cleanup --target ${{ matrix.warehouse }} | ||
- name: Run tests | ||
run: ./.scripts/integration_test.sh -d ${{ matrix.warehouse }} | ||
|
||
# post_ci_cleanup sits in utils package | ||
- name: "Post-test: Drop ci schemas" | ||
run: | | ||
dbt run-operation post_ci_cleanup --target ${{ matrix.warehouse }} |
Oops, something went wrong.