Skip to content

Commit

Permalink
fix: Issue with on_schema_change config
Browse files Browse the repository at this point in the history
- Moves the `raw_on_schema_change` variable back into scope for the
  config validator
- Adds `BaseIncrementalOnSchemaChange` test to test_incremental.py

resolves: #268
  • Loading branch information
maladroitthief committed Jan 29, 2025
1 parent 0e9c546 commit 68b9772
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 3 deletions.
7 changes: 5 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,13 @@

- When naming reflections, if a `name` config is not set, the `alias` config parameter will be used instead. If also undefined, it will refer to the model name instead of using `Unnamed Reflection`
- Grants can now be set for both users and for roles. A prefix was added to handle this, with `user:` and `role:` being the valid prefixes. For example, `user:dbt_test_user_1` and `role:dbt_test_role_1`. If no prefix is provided, defaults to user for backwards compatibility.
- Moves the `raw_on_schema_change` variable back into scope for the config validator
- Adds `BaseIncrementalOnSchemaChange` test to test_incremental.py

## Features

- [#259](https://github.com/dremio/dbt-dremio/pull/259) Added support for roles in grants
- [#273](https://github.com/dremio/dbt-dremio/pull/273) Fix issue with on_schema_change config

# dbt-dremio v1.8.1

Expand All @@ -32,7 +35,7 @@
- Integration via REST API

## Features

- [#250](https://github.com/dremio/dbt-dremio/pull/250) Implementation of wikis and tags feature
- [#250](https://github.com/dremio/dbt-dremio/pull/256) Reflections are now handled through the Rest API

Expand Down Expand Up @@ -60,7 +63,7 @@
## Changes

- [#199](https://github.com/dremio/dbt-dremio/issues/199) Populate PyPI's `long_description` with contents of `README.md`
- [#167](https://github.com/dremio/dbt-dremio/issues/167) Remove parentheses surrounding views in the create_view_as macro. In more complex queries, the parentheses cause performance issues.
- [#167](https://github.com/dremio/dbt-dremio/issues/167) Remove parentheses surrounding views in the create_view_as macro. In more complex queries, the parentheses cause performance issues.
- [#211](https://github.com/dremio/dbt-dremio/issues/211) Make fetching model data false by default. This improves performance where job results do not need to be populated.
- [#203](https://github.com/dremio/dbt-dremio/issues/203) Allow for dots in schema name, by surrounding in single and double quotes.
- [#193](https://github.com/dremio/dbt-dremio/issues/193) Fixes Reflection bug: The name argument to ref() must be a string, got <class 'jinja2.runtime.Undefined'>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ limitations under the License.*/
-- configs
{%- set unique_key = config.get('unique_key', validator=validation.any[list, basestring]) -%}
{%- set full_refresh_mode = (should_full_refresh()) -%}
{%- set raw_on_schema_change = config.get('on_schema_change', validator=validation.any[basestring]) or 'ignore' -%}
{%- set on_schema_change = incremental_validate_on_schema_change(raw_on_schema_change) -%}

-- the temp_ and backup_ relations should not already exist in the database; get_relation
Expand Down Expand Up @@ -70,7 +71,6 @@ limitations under the License.*/
{%- set file_format = dbt_dremio_validate_get_file_format(raw_file_format) -%}
{%- set incremental_predicates = config.get('predicates', none) or config.get('incremental_predicates', none) -%}
{%- set strategy = dbt_dremio_validate_get_incremental_strategy(incremental_strategy) -%}
{%- set raw_on_schema_change = config.get('on_schema_change', validator=validation.any[basestring]) or 'ignore' -%}
{% set build_sql = dbt_dremio_get_incremental_sql(strategy, intermediate_relation, target_relation, dest_columns, unique_key) %}

{% endif %}
Expand Down
4 changes: 4 additions & 0 deletions tests/functional/adapter/basic/test_incremental.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
from dbt.tests.adapter.basic.test_incremental import (
BaseIncremental,
BaseIncrementalNotSchemaChange,
BaseIncrementalOnSchemaChange,
)
from dbt.tests.adapter.incremental.test_incremental_merge_exclude_columns import (
BaseMergeExcludeColumns,
Expand Down Expand Up @@ -112,6 +113,9 @@ def test_incremental(self, project):
class TestBaseIncrementalNotSchemaChange(BaseIncrementalNotSchemaChange):
pass

class TestIncrementalOnSchemaChange(BaseIncrementalOnSchemaChange):
pass


class TestBaseMergeExcludeColumnsDremio(BaseMergeExcludeColumns):
def get_test_fields(self, project, seed, incremental_model, update_sql_file):
Expand Down

0 comments on commit 68b9772

Please sign in to comment.