dbt_fivetran_log v0.5.4

Happy Wednesday 🐫!

This release of the dbt_fivetran_log package includes the following updates:

Fixes

• The unique combination of columns test within the fivetran_log__schema_changelog model has been updated to also check the message_data field. This is needed as schema changelog events may now sync at the same time. (#51)
• The fivetran_log__connector_status model has been adjusted to filter out all logs that contain a transformation_id. Transformation logs are not always synced as a JSON object and thus the package may encounter errors on Snowflake warehouses when parsing non-JSON fields. Since transformation records are not used in this end model, they have been filtered out. (#51)

dbt_fivetran_log v0.5.3

Happy Wednesday 🐫

This release of the dbt_fivetran_log package includes the following updates:

Fixes

• Per the Fivetran Log December 2021 Release Notes every sync results in a final sync_end event. In the previous version of this package, a successful sync was identified via a sync_end event while anything else was a version of broken. Since all syncs result in a sync_end event now, the package has been updated to account for this change within the connector.
• To account for the above fix, a new field (last_successful_sync_completed_at) was added to the fivetran_log__connector_status model. This field captures the last successful sync for the connector.

dbt_fivetran_log 0.5.2

Happy Wednesday!

This release of the dbt_fivetran_log package includes the following updates:

Fixes

• The fivetran_log__connector_status model uses a date function off of created_at from the stg_fivetran_log__log model. This fails on certain redshift destinations as the timestamp is synced as timestamptz. Therefore, the field within the staging model is cast using dbt_utils.type_timestamp to appropriately cast the field for downstream functions. Further, to future proof, timestamps were cast within the following staging models: account, account_membership, active_volume, destination_membership, destination, log, transformation, and user. (#40)

dbt_fivetran_log 0.5.1

Features

This release just introduces Databricks compatibility! 🧱🧱

dbt_fivetran_log 0.5.0

🎉 Official dbt v1.0.0 Compatibility Release 🎉

🚨 Breaking Changes 🚨

• Adjusts the require-dbt-version to now be within the range [">=1.0.0", "<2.0.0"]. Additionally, the package has been updated for dbt v1.0.0 compatibility. If you are using a dbt version <1.0.0, you will need to upgrade in order to leverage the latest version of the package.
  • For help upgrading your package, I recommend reviewing this GitHub repo's Release Notes on what changes have been implemented since your last upgrade.
  • For help upgrading your dbt project to dbt v1.0.0, I recommend reviewing dbt-labs upgrading to 1.0.0 docs for more details on what changes must be made.
• Upgrades the package dependency to refer to the latest dbt_fivetran_utils. The latest dbt_fivetran_utils package also has a dependency on dbt_utils [">=0.8.0", "<0.9.0"].
  • Please note, if you are installing a version of dbt_utils in your packages.yml that is not in the range above then you will encounter a package dependency error.

Additional Features

• Materializes the fivetran_log__audit_table incrementally, and employs partitioning for BigQuery users. As a non-incremental table, this model involved high runtimes for some users (#27)
  • If you would like to apply partitioning to the underlying source tables (ie the LOG table), refer to Fivetran docs on how to do so.
• Expands compatibility to Postgres!

Thanks to @b-luu for contributing! (#30)

dbt_fivetran_log 0.5.0-b1

🎉 dbt v1.0.0 Compatibility Pre Release 🎉 An official dbt v1.0.0 compatible version of the package will be released once existing feature/bug PRs are merged.

🚨 Breaking Changes 🚨

• Adjusts the require-dbt-version to now be within the range [">=1.0.0", "<2.0.0"]. Additionally, the package has been updated for dbt v1.0.0 compatibility. If you are using a dbt version <1.0.0, you will need to upgrade in order to leverage the latest version of the package.
  • For help upgrading your package, I recommend reviewing this GitHub repo's Release Notes on what changes have been implemented since your last upgrade.
  • For help upgrading your dbt project to dbt v1.0.0, I recommend reviewing dbt-labs upgrading to 1.0.0 docs for more details on what changes must be made.
• Upgrades the package dependency to refer to the latest dbt_fivetran_utils. The latest dbt_fivetran_utils package also has a dependency on dbt_utils [">=0.8.0", "<0.9.0"].
  • Please note, if you are installing a version of dbt_utils in your packages.yml that is not in the range above then you will encounter a package dependency error.

Include Priority First Syncs as Connector Health Statuses

Happy Monday!

This release addresses issue #23, and incorporates logic into the fivetran_log__connector_status model to accommodate the way priority-first syncs are currently logged. Priority first sync is now included as a potential connector_health value.

More details in the CHANGELOG.

dbt 0.20.0 Compatibility

🎉 dbt 0.20.0 Compatibility 🎉

🚨 This is a breaking change! 🚨 dbt v0.20.0 or greater is required for this release. If you are not ready to upgrade, consider using a previous release of this package.

Additional considerations when upgrading to this package:

• This package utilizes the latest v0.7.0 release of the dbt-labls/dbt_utils package. If your project also utilizes a version of the fishtown-analytics/dbt_utils package then you will receive a duplicate package error. If this is the case you will need to consider upgrading your other packages to be compatible with this update or use a previous release of this package.
• Similar to the above point, all previous Fivetran dbt packages utilize the fishtown-analytics/dbt_utils package and you will need to upgrade all Fivetran dbt packages to the latest dbt 0.20.0 compatibility releases in order for your packages to run without package conflicts.
• The addition of the fivetran_log_using_sync_alert_messages variable to allow users to disable the Fivetran error and warning alerts from the fivetran_log__connector_status model as some users experienced list_agg errors if the messages were too long.

Fivetran Utils Dependency Update

🚧 Fivetran Utils Dependency Update 🚧

This release includes the following non-breaking changes:

• Update to the packages.yml to reference the recently released Fivetran Utils package. This is a necessary update to ensure this package is compatible with previous dbt releases before dbt v0.20.0 updates take place.

Connector Type ID Bug Fix

This release includes the following non breaking changes:

• Resolution to Issue #14 by adding the get_connector_columns macro and stg_fivetran_log__connector_tmp model to the package in order to reflect the connector level change that resulted in removing the connector_type field and now only contains connector_type_id.