Skip to content

Commit

Permalink
Merge branch '1.8.latest' into patch-1
Browse files Browse the repository at this point in the history
  • Loading branch information
mikealfare authored Nov 9, 2024
2 parents c5af5d4 + a6e665b commit 19071df
Show file tree
Hide file tree
Showing 21 changed files with 259 additions and 66 deletions.
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 1.8.2
current_version = 1.8.4
parse = (?P<major>[\d]+) # major version number
\.(?P<minor>[\d]+) # minor version number
\.(?P<patch>[\d]+) # patch version number
Expand Down
5 changes: 5 additions & 0 deletions .changes/1.8.3.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
## dbt-snowflake 1.8.3 - June 12, 2024

### Fixes

- return to previous naming convention to return to quoting policy ([#1074](https://github.com/dbt-labs/dbt-snowflake/issues/1074))
5 changes: 5 additions & 0 deletions .changes/1.8.4.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
## dbt-snowflake 1.8.4 - October 17, 2024

### Fixes

- Fix scenario where using the `--empty` flag causes metadata queries to contain limit clauses ([#1033](https://github.com/dbt-labs/dbt-snowflake/issues/1033))
6 changes: 6 additions & 0 deletions .changes/unreleased/Breaking Changes-20241016-183143.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Breaking Changes
body: Drop support for Python 3.8
time: 2024-10-16T18:31:43.4167-04:00
custom:
Author: mikealfare
Issue: "1211"
8 changes: 4 additions & 4 deletions .github/scripts/integration-test-matrix.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module.exports = ({ context }) => {
const defaultPythonVersion = "3.8";
const supportedPythonVersions = ["3.8", "3.9", "3.10", "3.11"];
const defaultPythonVersion = "3.9";
const supportedPythonVersions = ["3.9", "3.10", "3.11"];
const supportedAdapters = ["snowflake"];

// if PR, generate matrix based on files changed and PR labels
Expand Down Expand Up @@ -44,7 +44,7 @@ module.exports = ({ context }) => {

if (labels.includes("test macos") || testAllLabel) {
include.push({
os: "macos-12",
os: "macos-14",
adapter,
"python-version": pythonVersion,
});
Expand Down Expand Up @@ -78,7 +78,7 @@ module.exports = ({ context }) => {
// additionally include runs for all adapters, on macos and windows,
// but only for the default python version
for (const adapter of supportedAdapters) {
for (const operatingSystem of ["windows-latest", "macos-12"]) {
for (const operatingSystem of ["windows-latest", "macos-14"]) {
include.push({
os: operatingSystem,
adapter: adapter,
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.8'
python-version: '3.9'

- name: Install python dependencies
run: |
Expand All @@ -72,7 +72,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11']
python-version: ['3.9', '3.10', '3.11']

env:
TOXENV: "unit"
Expand Down Expand Up @@ -127,7 +127,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.8'
python-version: '3.9'

- name: Install python dependencies
run: |
Expand Down Expand Up @@ -173,8 +173,8 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-12, windows-latest]
python-version: ['3.8', '3.9', '3.10', '3.11']
os: [ubuntu-latest, macos-14, windows-latest]
python-version: ['3.9', '3.10', '3.11']

steps:
- name: Set up Python ${{ matrix.python-version }}
Expand Down
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ repos:
additional_dependencies: ['click~=8.1']
args:
- "--line-length=99"
- "--target-version=py38"
- "--target-version=py39"
- id: black
alias: black-check
stages: [manual]
additional_dependencies: ['click~=8.1']
args:
- "--line-length=99"
- "--target-version=py38"
- "--target-version=py39"
- "--check"
- "--diff"
- repo: https://github.com/pycqa/flake8
Expand Down
16 changes: 14 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,25 @@
- "Breaking changes" listed under a version may require action from end users or external maintainers when upgrading to that version.
- Do not edit this file directly. This file is auto-generated using [changie](https://github.com/miniscruff/changie). For details on how to document a change, see [the contributing guide](https://github.com/dbt-labs/dbt-snowflake/blob/main/CONTRIBUTING.md#adding-changelog-entry)

## dbt-snowflake 1.8.2 - May 23, 2024
## dbt-snowflake 1.8.4 - October 17, 2024

### Fixes

- Update relation caching to correctly identify dynamic tables, accounting for Snowflake's `2024_03` bundle ([#1016](https://github.com/dbt-labs/dbt-snowflake/issues/1016))
- Fix scenario where using the `--empty` flag causes metadata queries to contain limit clauses ([#1033](https://github.com/dbt-labs/dbt-snowflake/issues/1033))



## dbt-snowflake 1.8.3 - June 12, 2024

### Fixes

- return to previous naming convention to return to quoting policy ([#1074](https://github.com/dbt-labs/dbt-snowflake/issues/1074))

## dbt-snowflake 1.8.2 - May 23, 2024

### Fixes

- Update relation caching to correctly identify dynamic tables, accounting for Snowflake's `2024_03` bundle ([#1016](https://github.com/dbt-labs/dbt-snowflake/issues/1016))

## dbt-snowflake 1.8.1 - May 15, 2024

Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ WARNING: The parameters in your `test.env` file must link to a valid Snowflake a
There are a few methods for running tests locally.

#### `tox`
`tox` automatically runs unit tests against several Python versions using its own virtualenvs. Run `tox -p` to run unit tests for Python 3.8, Python 3.9, Python 3.10, and `flake8` in parallel. Run `tox -e py38` to invoke tests on Python version 3.8 only (use py38, py39, or py310). Tox recipes are found in `tox.ini`.
`tox` automatically runs unit tests against several Python versions using its own virtualenvs. Run `tox -p` to run unit tests for Python 3.9 and Python 3.10, and `flake8` in parallel. Run `tox -e py39` to invoke tests on Python version 3.9 only (use py39 or py310). Tox recipes are found in `tox.ini`.

#### `pytest`
You may run a specific test or group of tests using `pytest` directly. Activate a Python virtualenv active with dev dependencies installed. Then, run tests like so:
Expand Down
12 changes: 6 additions & 6 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -38,22 +38,22 @@ linecheck: ## Checks for all Python lines 100 characters or more
find dbt -type f -name "*.py" -exec grep -I -r -n '.\{100\}' {} \;

.PHONY: unit
unit: ## Runs unit tests with py38.
unit: ## Runs unit tests with py39.
@\
tox -e py38
tox -e py39

.PHONY: test
test: ## Runs unit tests with py38 and code checks against staged changes.
test: ## Runs unit tests with py39 and code checks against staged changes.
@\
tox -p -e py38; \
tox -p -e py39; \
pre-commit run black-check --hook-stage manual | grep -v "INFO"; \
pre-commit run flake8-check --hook-stage manual | grep -v "INFO"; \
pre-commit run mypy-check --hook-stage manual | grep -v "INFO"

.PHONY: integration
integration: ## Runs snowflake integration tests with py38.
integration: ## Runs snowflake integration tests with py39.
@\
tox -e py38-snowflake --
tox -e py39-snowflake --

.PHONY: clean
@echo "cleaning repo"
Expand Down
2 changes: 1 addition & 1 deletion dbt/adapters/snowflake/__version__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
version = "1.8.2"
version = "1.8.4"
43 changes: 27 additions & 16 deletions dbt/include/snowflake/macros/adapters.sql
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,14 @@

{% macro snowflake__get_columns_in_relation(relation) -%}
{%- set sql -%}
describe table {{ relation }}
describe table {{ relation.render() }}
{%- endset -%}
{%- set result = run_query(sql) -%}

{% set maximum = 10000 %}
{% if (result | length) >= maximum %}
{% set msg %}
Too many columns in relation {{ relation }}! dbt can only get
Too many columns in relation {{ relation.render() }}! dbt can only get
information about relations with fewer than {{ maximum }} columns.
{% endset %}
{% do exceptions.raise_compiler_error(msg) %}
Expand Down Expand Up @@ -72,9 +72,15 @@

{% for _ in range(0, max_iter) %}

{%- set paginated_sql -%}
show objects in {{ schema_relation.database }}.{{ schema_relation.schema }} limit {{ max_results_per_iter }} from '{{ watermark.table_name }}'
{%- endset -%}
{% if schema_relation is string %}
{%- set paginated_sql -%}
show objects in {{ schema_relation }} limit {{ max_results_per_iter }} from '{{ watermark.table_name }}'
{%- endset -%}
{% else %}
{%- set paginated_sql -%}
show objects in {{ schema_relation.include(identifier=False) }} limit {{ max_results_per_iter }} from '{{ watermark.table_name }}'
{%- endset -%}
{% endif -%}

{%- set paginated_result = run_query(paginated_sql) %}
{%- set paginated_n = (paginated_result | length) -%}
Expand All @@ -96,7 +102,7 @@

{%- if loop.index == max_iter -%}
{%- set msg -%}
dbt will list a maximum of {{ max_total_results }} objects in schema {{ schema_relation.database }}.{{ schema_relation.schema }}.
dbt will list a maximum of {{ max_total_results }} objects in schema {{ schema_relation }}.
Your schema exceeds this limit. Please contact support@getdbt.com for troubleshooting tips,
or review and reduce the number of objects contained.
{%- endset -%}
Expand All @@ -122,10 +128,15 @@
{% macro snowflake__list_relations_without_caching(schema_relation, max_iter=10, max_results_per_iter=10000) %}

{%- set max_total_results = max_results_per_iter * max_iter -%}

{%- set sql -%}
show objects in {{ schema_relation.database }}.{{ schema_relation.schema }} limit {{ max_results_per_iter }}
{%- endset -%}
{% if schema_relation is string %}
{%- set sql -%}
show objects in {{ schema_relation }} limit {{ max_results_per_iter }}
{%- endset -%}
{% else %}
{%- set sql -%}
show objects in {{ schema_relation.include(identifier=False) }} limit {{ max_results_per_iter }}
{%- endset -%}
{% endif -%}

{%- set result = run_query(sql) -%}

Expand Down Expand Up @@ -166,7 +177,7 @@

{% macro snowflake__alter_column_type(relation, column_name, new_column_type) -%}
{% call statement('alter_column_type') %}
alter table {{ relation }} alter {{ adapter.quote(column_name) }} set data type {{ new_column_type }};
alter table {{ relation.render() }} alter {{ adapter.quote(column_name) }} set data type {{ new_column_type }};
{% endcall %}
{% endmacro %}

Expand All @@ -176,7 +187,7 @@
{%- else -%}
{%- set relation_type = relation.type -%}
{%- endif -%}
comment on {{ relation_type }} {{ relation }} IS $${{ relation_comment | replace('$', '[$]') }}$$;
comment on {{ relation_type }} {{ relation.render() }} IS $${{ relation_comment | replace('$', '[$]') }}$$;
{% endmacro %}


Expand All @@ -187,7 +198,7 @@
{% else -%}
{% set relation_type = relation.type %}
{% endif %}
alter {{ relation_type }} {{ relation }} alter
alter {{ relation_type }} {{ relation.render() }} alter
{% for column_name in existing_columns if (column_name in existing_columns) or (column_name|lower in existing_columns) %}
{{ get_column_comment_sql(column_name, column_dict) }} {{- ',' if not loop.last else ';' }}
{% endfor %}
Expand Down Expand Up @@ -246,7 +257,7 @@
{% if add_columns %}

{% set sql -%}
alter {{ relation_type }} {{ relation }} add column
alter {{ relation_type }} {{ relation.render() }} add column
{% for column in add_columns %}
{{ column.name }} {{ column.data_type }}{{ ',' if not loop.last }}
{% endfor %}
Expand All @@ -259,7 +270,7 @@
{% if remove_columns %}

{% set sql -%}
alter {{ relation_type }} {{ relation }} drop column
alter {{ relation_type }} {{ relation.render() }} drop column
{% for column in remove_columns %}
{{ column.name }}{{ ',' if not loop.last }}
{% endfor %}
Expand Down Expand Up @@ -292,7 +303,7 @@

{% macro snowflake__truncate_relation(relation) -%}
{% set truncate_dml %}
truncate table {{ relation }}
truncate table {{ relation.render() }}
{% endset %}
{% call statement('truncate_relation') -%}
{{ snowflake_dml_explicit_transaction(truncate_dml) }}
Expand Down
2 changes: 1 addition & 1 deletion dev-requirements.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# install latest changes in dbt-core
# TODO: how to automate switching from develop to version branches?
git+https://github.com/dbt-labs/dbt-core.git@1.8.latest#egg=dbt-core&subdirectory=core
git+https://github.com/dbt-labs/dbt-core.git#egg=dbt-core&subdirectory=core
git+https://github.com/dbt-labs/dbt-adapters.git
git+https://github.com/dbt-labs/dbt-adapters.git#subdirectory=dbt-tests-adapter

Expand Down
8 changes: 3 additions & 5 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
import sys

# require python 3.8 or newer
if sys.version_info < (3, 8):
if sys.version_info < (3, 9):
print("Error: dbt does not support this version of Python.")
print("Please upgrade to Python 3.8 or higher.")
print("Please upgrade to Python 3.9 or higher.")
sys.exit(1)


Expand Down Expand Up @@ -72,11 +72,9 @@ def _plugin_version() -> str:
"Operating System :: Microsoft :: Windows",
"Operating System :: MacOS :: MacOS X",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
],
python_requires=">=3.8",
python_requires=">=3.9",
)
Loading

0 comments on commit 19071df

Please sign in to comment.