Skip to content

Commit

Permalink
fix: Issue with on_schema_change config
Browse files Browse the repository at this point in the history
- Moves the `raw_on_schema_change` variable back into scope for the
  config validator
- Adds `BaseIncrementalOnSchemaChange` test to test_incremental.py

resolves: #268
  • Loading branch information
maladroitthief committed Jan 29, 2025
1 parent 0e9c546 commit b06e889
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 3 deletions.
15 changes: 13 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,14 @@
# dbt-dremio v1.8.3

## Changes

- Moves the `raw_on_schema_change` variable back into scope for the config validator
- Adds `BaseIncrementalOnSchemaChange` test to test_incremental.py

## Features

- [#273](https://github.com/dremio/dbt-dremio/pull/273) Fix issue with on_schema_change config

# dbt-dremio v1.8.2

## Changes
Expand Down Expand Up @@ -32,7 +43,7 @@
- Integration via REST API

## Features

- [#250](https://github.com/dremio/dbt-dremio/pull/250) Implementation of wikis and tags feature
- [#250](https://github.com/dremio/dbt-dremio/pull/256) Reflections are now handled through the Rest API

Expand Down Expand Up @@ -60,7 +71,7 @@
## Changes

- [#199](https://github.com/dremio/dbt-dremio/issues/199) Populate PyPI's `long_description` with contents of `README.md`
- [#167](https://github.com/dremio/dbt-dremio/issues/167) Remove parentheses surrounding views in the create_view_as macro. In more complex queries, the parentheses cause performance issues.
- [#167](https://github.com/dremio/dbt-dremio/issues/167) Remove parentheses surrounding views in the create_view_as macro. In more complex queries, the parentheses cause performance issues.
- [#211](https://github.com/dremio/dbt-dremio/issues/211) Make fetching model data false by default. This improves performance where job results do not need to be populated.
- [#203](https://github.com/dremio/dbt-dremio/issues/203) Allow for dots in schema name, by surrounding in single and double quotes.
- [#193](https://github.com/dremio/dbt-dremio/issues/193) Fixes Reflection bug: The name argument to ref() must be a string, got <class 'jinja2.runtime.Undefined'>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ limitations under the License.*/
-- configs
{%- set unique_key = config.get('unique_key', validator=validation.any[list, basestring]) -%}
{%- set full_refresh_mode = (should_full_refresh()) -%}
{%- set raw_on_schema_change = config.get('on_schema_change', validator=validation.any[basestring]) or 'ignore' -%}
{%- set on_schema_change = incremental_validate_on_schema_change(raw_on_schema_change) -%}

-- the temp_ and backup_ relations should not already exist in the database; get_relation
Expand Down Expand Up @@ -70,7 +71,6 @@ limitations under the License.*/
{%- set file_format = dbt_dremio_validate_get_file_format(raw_file_format) -%}
{%- set incremental_predicates = config.get('predicates', none) or config.get('incremental_predicates', none) -%}
{%- set strategy = dbt_dremio_validate_get_incremental_strategy(incremental_strategy) -%}
{%- set raw_on_schema_change = config.get('on_schema_change', validator=validation.any[basestring]) or 'ignore' -%}
{% set build_sql = dbt_dremio_get_incremental_sql(strategy, intermediate_relation, target_relation, dest_columns, unique_key) %}

{% endif %}
Expand Down
4 changes: 4 additions & 0 deletions tests/functional/adapter/basic/test_incremental.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
from dbt.tests.adapter.basic.test_incremental import (
BaseIncremental,
BaseIncrementalNotSchemaChange,
BaseIncrementalOnSchemaChange,
)
from dbt.tests.adapter.incremental.test_incremental_merge_exclude_columns import (
BaseMergeExcludeColumns,
Expand Down Expand Up @@ -112,6 +113,9 @@ def test_incremental(self, project):
class TestBaseIncrementalNotSchemaChange(BaseIncrementalNotSchemaChange):
pass

class TestIncrementalOnSchemaChange(BaseIncrementalOnSchemaChange):
pass


class TestBaseMergeExcludeColumnsDremio(BaseMergeExcludeColumns):
def get_test_fields(self, project, seed, incremental_model, update_sql_file):
Expand Down

0 comments on commit b06e889

Please sign in to comment.