diff --git a/CHANGELOG b/CHANGELOG index 055441ab..4adf0c73 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -7,6 +7,9 @@ XXX - Add new passthrough aggregations to the views, sessions, and users table, enabled using `snowplow__view/session/user_aggregations` - Reorder and add some additional context fields to derived tables (non-breaking change) +## Fixes +- Fix a bug where if you ran the package in a period with no data, and had list all events enabled, the package would error rather than complete + ## Under the hood - Prefix all macro calls with package name for easier customization - Use macros for grouped fields (e.g. contexts) where possible diff --git a/integration_tests/.scripts/integration_test.sh b/integration_tests/.scripts/integration_test.sh index 41c3e979..54062b32 100755 --- a/integration_tests/.scripts/integration_test.sh +++ b/integration_tests/.scripts/integration_test.sh @@ -26,6 +26,10 @@ for db in ${DATABASES[@]}; do echo "Snowplow unified integration tests: Seeding data" eval "dbt seed --full-refresh --target $db" || exit 1; + echo "Snowplow unified integration tests: Try run without data" + eval "dbt run --full-refresh --vars '{snowplow__allow_refresh: true, snowplow__backfill_limit_days: 1, snowplow__enable_cwv: false, snowplow__start_date: 2010-01-01}' --target $db" || exit 1; + + echo "Snowplow unified integration tests: Conversions" eval "dbt run --full-refresh --select +snowplow_unified_conversions snowplow_unified_integration_tests.source --vars '{snowplow__allow_refresh: true, snowplow__backfill_limit_days: 220, snowplow__enable_cwv: false, snowplow__enable_conversions: true}' --target $db" || exit 1; diff --git a/macros/field_definitions/event_counts_string_query.sql b/macros/field_definitions/event_counts_string_query.sql index 5625e08e..ae63fd1f 100644 --- a/macros/field_definitions/event_counts_string_query.sql +++ b/macros/field_definitions/event_counts_string_query.sql @@ -10,7 +10,7 @@ You may obtain a copy of the Snowplow Personal and Academic License Version 1.0 {% endmacro %} {% macro default__event_counts_string_query() %} - {% set event_names = dbt_utils.get_column_values(ref('snowplow_unified_events_this_run'), 'event_name', order_by = 'event_name') %} + {% set event_names = dbt_utils.get_column_values(ref('snowplow_unified_events_this_run'), 'event_name', order_by = 'event_name', default = []) %} {# Loop over every event_name in this run, create a json string of the name and count ONLY if there are events with that name in the session (otherwise empty string), then trim off the last comma (cannot use loop.first/last because first/last entry may not have any events for that session) #} @@ -24,7 +24,7 @@ You may obtain a copy of the Snowplow Personal and Academic License Version 1.0 {% endmacro %} {% macro bigquery__event_counts_string_query() %} - {% set event_names = dbt_utils.get_column_values(ref('snowplow_unified_events_this_run'), 'event_name', order_by = 'event_name') %} + {% set event_names = dbt_utils.get_column_values(ref('snowplow_unified_events_this_run'), 'event_name', order_by = 'event_name', default = []) %} {# Loop over every event_name in this run, create a json string of the name and count ONLY if there are events with that name in the session (otherwise empty string), then trim off the last comma (cannot use loop.first/last because first/last entry may not have any events for that session) #} @@ -39,18 +39,22 @@ You may obtain a copy of the Snowplow Personal and Academic License Version 1.0 {% macro spark__event_counts_string_query() %} - {% set event_names = dbt_utils.get_column_values(ref('snowplow_unified_base_events_this_run'), 'event_name', order_by = 'event_name') %} + {% set event_names = dbt_utils.get_column_values(ref('snowplow_unified_base_events_this_run'), 'event_name', order_by = 'event_name', default = []) %} {# Loop over every event_name in this run, create a map of the name and count, later filter for only events with that name in the session #} - map( - {%- for event_name in event_names %} - '{{event_name}}', sum(case when event_name = '{{event_name}}' then 1 else 0 end){% if not loop.last %},{% endif %} - {%- endfor -%} - ) + {% if event_names %} + map( + {%- for event_name in event_names %} + '{{event_name}}', sum(case when event_name = '{{event_name}}' then 1 else 0 end){% if not loop.last %},{% endif %} + {%- endfor -%} + ) + {% else %} + cast(null as map) + {% endif %} {% endmacro %} {% macro postgres__event_counts_string_query() %} - {% set event_names = dbt_utils.get_column_values(ref('snowplow_unified_events_this_run'), 'event_name', order_by = 'event_name') %} + {% set event_names = dbt_utils.get_column_values(ref('snowplow_unified_events_this_run'), 'event_name', order_by = 'event_name', default = []) %} {# Loop over every event_name in this run, create a json string of the name and count ONLY if there are events with that name in the session (otherwise empty string), then trim off the last comma (cannot use loop.first/last because first/last entry may not have any events for that session) #}