Skip to content

Releases: data-solution-automation-engine/TEAM

TEAM v1.6.17

02 Sep 23:22
Compare
Choose a tag to compare

New features

  • Extensions at Data Object Mapping level are now editable/visible in the main grid

Breaking changes

  • N/A

Bug fixes

  • Various minor issue that caused validation to stop. This is now better managed via exception handling
  • Colour coding caused a hang-up in the grid display in some cases, this is now fixed

Known issues

  • High DPI monitors (e.g. 150 scaling) still causes issues for connection and environment tabs. Somehow this is not picked up by the DPI scale mode still

TEAM v1.6.15

08 Jul 22:35
Compare
Choose a tag to compare

New features

  • Additional support for Data Queries for Business Key Definitions. The schema for data solution automation stipulates that an IDataItem can be a query (logic) or a column/attribute. This was not yet supported, now you can use `` to indicate that a business key, or business key component is a logic component. This will resolve into a Data Query. For example: COMPOSITE(CUSTOMER_ID;SELECT 'Yolo') will now create a composite of a Data Item and separate Data Query.

Breaking changes

  • N/A

Bug fixes

  • Various minor issues, added additional exception handling. For instance showing the root cause when a Snowflake Warehouse is not available.
  • Some better handling of screen resizing, improving the performance of the filters etc.

TEAM v1.6.14

04 Mar 05:01
Compare
Choose a tag to compare

New features

  • Additional support for using Snowflake Stored Procedures and Functions as source data objects.
  • Added a new JSON export setting 'objectType' to provide a place to distinguish between, well, object types (e.g. table, view, procedure) so this can be differentiated in the templates.
  • Additional Snowflake query support, running multiple statements in one go.
  • Unselect any connections when not-relevant for the in-scope metadata (to simplify / speed-up reverse-engineering).

Breaking changes

  • N/A

Bug fixes

  • Various minor issues, exception handling.

TEAM v1.6.13

12 Nov 18:26
209d9a9
Compare
Choose a tag to compare

New features

  • Snowflake connection feature, including separate form to create new connection and support for reverse-engineering.
  • Additional improvements on high-DPI support, this should address various issues around overlapping controls on monitors with 125% or 150% scaling.

Breaking changes

N/A

Bug fixes

N/A

TEAM v1.6.12

23 Aug 23:28
Compare
Choose a tag to compare

New features

  • Clear event log feature.
  • Reported errors during various processing steps are now also shown in the main screen. For example, when parsing you'll see an error if something fails - not just in the event log.
  • Made the form resize a bit more generous, so it can grow smaller.
  • New feature to override reserved column names in data item mappings, for example EFFECTIVE_DATETIME (or equivalent) can be mapped as well.
  • Refresh metadata menu option, for when underlying JSON files are updated outside of TEAM.

Breaking changes

N/A

Bug fixes

  • Fixed a bug that incorrectly retained the schema name after updating a data object mapping.
  • Fixed a bug that showed unrelated mappings when using a filter and adding mappings.
  • Filter is now correctly maintained in regular actions, for example reverse-engineering, adding mappings etc.
  • Fixed a bug where the data object mapping import feature did not correctly identify some business key syntax.
  • Fixed a bug where a mapping level classification was not always retained.
  • Cleaned up event log, so no duplicate entries are added.

TEAM v.1.6.10

28 Jul 03:11
Compare
Choose a tag to compare

New features

  • Better handling of schemas. The data grid will always interpret schemas, but only show the schema where it differs from the default schema that is defined for the corresponding connection. Only if the schema is different, it will be visible in the data grid. In all cases will the schema be saved as an extension.
  • You can now modify data object mapping classifications from TEAM, so that it can automatically be linked to certain templates in VDW.
  • Extended query timeout to 10 minutes, in some (rare) cases, the reverse-engineering would time out because the retrieval of the physical model snapshot took too long.
  • Created a support directory upon install with some useful scripts.

Breaking changes

  • Not really a breaking change, but if the export feature to add the 'schema' to the connection extension is enabled, this will also be added at data object level. This is for done for (future) backward-compatibility. Conceptually, this is a property of the data object, not the connection.

Bug fixes

  • Fixed a bug that prevented adding data items to the JSON file when the physical model is not yet available (to get the data type). Now, it will be added without a data type, because this is not known yet.
  • Typo in Target Data Item column header for the data item grid.
  • Fixed a bug that overwrote mapping-level classification changes.
  • Fixed a bug that overwrote mapping-level extensions.
  • Fixed a bug where data objects with the same name, but different connections were considered the same.
  • Fixed a bug where the connection was not loaded correctly (because of JSON syntax errors made by users/outside of TEAM) but didn't notify the user.

TEAM v1.6.9

23 Mar 02:20
Compare
Choose a tag to compare

New features

  • Major revision of out of the box samples, to contain additional edge cases and bitemporal examples
  • Alignment of default data solution columns to standards
  • Ability to use the full row context menu to update the JSON structure directly, for example for adding data object mapping level extensions
  • Various menu options to access the 'core' directory, to update the environments file when required
  • Upgraded all references, including latest DataWarehouseAutomation library
  • Upgraded solution to .Net 7 (to Core from Framework 4.8)
  • Extended TeamEnvironment to store VDW information for easy environment switching in both tools
  • Connections that are not evaluated to be of a predefined type/convention now default to a connection name classification, not 'source'
  • DGML export now supports filtered data object mappings

Breaking changes

  • If relying on a data object classification 'source' which is not a source system, this may change to the classification
  • Samples would have to be re-generated, if used

Bug fixes

  • Fixed DGML export, now works based on the JSON files including references
  • Fixed bug where a Link did not correctly evaluate the referencing Hubs in a Data Vault deployment
  • Added additional reverse engineering logging options and removed unhandled exceptions
  • Fixed bug that caused a 'cannot find targetDataObjectname' when parsing from data item or physical model grid view
  • Fixed bug that caused unhandled exception when previewing a filtered row in the data object grid

TEAM v.1.6.8

31 Dec 05:32
Compare
Choose a tag to compare

New features

  • Improved the way the physical model snapshot is maintained as a JSON structure. For very large solutions it can become cumbersome to manage this in a single file. TEAM v1.6.8 changed this so that each table or file is saved in JSON as its own file. These files are organised in a <connection/database> / <schema/container> / file directory structure. Having separate files for each data object makes merging changes via Git much easier
  • Added an import feature for (now legacy) tabular physical model JSON files. Loading a file now creates the contents as separate files in the aforementioned structure
  • Modified the filter criterion to be case-insensitive everywhere (this only applied to the mapping grids in the previous version)
  • Pressing 'end' in the data item mapping grid will now enter the cell (edit mode) and navigate to the end of the value in that cell. This is an improvement when editing the grid in many places
  • Added full-row duplicate check for the data item mappings. If a duplicate is present in the grid, the validation process will report this (if duplicate detection is enabled in the validator form)

Breaking changes

  • TEAM v1.6.8 will expect a 'PhysicalModel' directory in the designated metadata directory, and create this if it does not exist. The physical model grid will use this directory to manage the physical model snapshot. Previous versions, that use the 'TEAM_Model_Metadata.json' single file will initially not see the physical model, because TEAM expects the separate data object JSON files to be present in the PhysicalModel directory. To convert, please import the TEAM_Model_Metadata.json file using the Physical Model Import feature, available in the menu. This only needs to be done once, after which the TEAM_Model_Metadata.json is no longer required.

Bug fixes

  • Updated the hash key approach for data item mappings to be truly unique for each row in the data item mapping grid. Previously, a hash value was issued upon creation. In some cases, when many records are created the same way in a bulk process and later completed, this would result in multiple mappings having the same hash key. An update in one would then be applied everywhere. This fix will prevent this from ever happening again
  • Fixed issue where the grid view would sometimes not render if an incorrectly structures JSON file was in one of the metadata sub-directories
  • Handled exception where the 'previous data object name' was not found during bulk-parsing

TEAM v1.6.7

13 Dec 09:43
Compare
Choose a tag to compare

New features

  • Enabling saving configuration settings per tab, as well as overall. CTRL-S now saves the current tab settings, and Shift-CTRL-S performs a 'save all'. Corresponding menu options have been added also
  • Filter is now case-insensitive
  • Data item mappings now have the same context menu options as the data object mapping
  • Added a JSON output setting that allows generating the Driving Key as an extension to the business key segment
  • Added as JSON output setting that allows generating the 'parent' (referencing) object as RelatedDataObject, e.g. the Hub for a Satellite or the Link for an Link-Satellite
  • Added validation on Driving Keys being syntactically correct
  • Added guard to prevent saving when there are row-level validations yet unresolved
  • Ability to provide a comma-separated list of exception columns that are ignored when creating data item mappings

Breaking changes

  • The naming for the JSON output settings have changed for consistency, the first time TEAM detects this it will show a warning asking the user to update the output settings. If you see this, go to the output settings, check the correct values and save for this message to disappear going forward

Bug fixes

  • Fixed issue where filter was removed when sorting the grids
  • Fixed issue where filter was removed after reverse-engineering
  • Pasting into a new row, or rows, now does not show an exception message, but creates new rows in the grid as per the number of pasted rows
  • Fixed issue where the parsing did not have full access to the physical model snapshot because of existing filters. Now, parse will always be able to access the full physical model so that automatically mapped data items appear correctly
  • Fixed issue where the Staging and Persistent Staging business key definitions created a business key data item mapping based on ordinal position. This was based on the assumption the first column(s) is/are the source primary keys but that is not always the case. Now the data item mapping is created so that the target data item is always the same as the source data item
  • Improved auto-resizing and scaling for various controls so that these work on smaller screens and/or higher scaling as well. Scrollbars are now added when controls can't fit on the screen
  • Fixed filtering on the physical grid which only applied a filter from the start of the string. Now does an equivalent of 'contains' so that the filter will return objects that contain the search string somewhere
  • Fixed issue where the first new entry in a data item mapping grid threw an exception
  • Fixed a bug where Multi-Active-Key classifications were not applied for automatically mapped data items
  • Reverse engineering with custom scripts now has meaningful exception handling
  • Fixed issue with tables not picked up during reverse-engineering using custom script connection
  • Handled exception related to reverse engineering unsaved changes
  • Fixed issue with the file paths not showing correctly in the configuration screen in some cases
  • Fixed issue when initial environment file does not contain configuration and metadata path

TEAM v1.6.6

18 Nov 03:29
f317215
Compare
Choose a tag to compare

The following changes / improvements have been made for TEAM v1.6.6:

  • Performance improvements across all grids, performance is good with 1000s of rows now
  • Improved filtering, also by adding a 'hide staging layer' checkbox that removes any staging, PSA mappings from view
  • Added duplicate mapping validator
  • Reverse-engineer, saving, validation and export now takes filters into account
  • Removed option to save / export without the JSON connection segment, this is a mandatory segment for TEAM. Previously, this has led to information getting lost
  • Added single / embedded executable for environments where MSI installation is not possible
  • Removed timestamps on saved configuration files, making the Git merge a bit easier
  • Added custom query feature for loading physical model metadata from somewhere else than a (SQL Server) data dictionary
  • Added feature to specify the definition of the Surrogate Key, so that you can define a more flexible convention for this
  • Added feature to explicitly specify exception columns that should not be added to data item mappings

What's Changed