Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop #587

Merged
merged 22 commits into from
Dec 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/R_CMD_check_Hades.yml
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ jobs:

- name: Upload source package
if: success() && runner.os == 'macOS' && github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
with:
name: package_tarball
path: check/*.tar.gz
Expand Down
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Package: DataQualityDashboard
Type: Package
Title: Execute and View Data Quality Checks on OMOP CDM Database
Version: 2.6.1
Date: 2024-07-12
Version: 2.6.2
Date: 2024-12-22
Authors@R: c(
person("Katy", "Sadowski", email = "sadowski@ohdsi.org", role = c("aut", "cre")),
person("Clair", "Blacketer", role = c("aut")),
Expand Down
23 changes: 22 additions & 1 deletion NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,24 @@
DataQualityDashboard 2.6.2
==========================
This release includes:

### Bugfixes

- Some fields were missing a standardConceptFieldName for the sourceValueCompleteness check, causing those checks to fail with an error. The missing field names have now been added
- Many unit plausibility checks were missing legitimate units from the list of plausible units - these have now been updated
- Sample results files in the shinyApps folder which had been causing issues for some users in rendering the Shiny app have now been removed
- The fkDomain check has been disabled for the episode.episode_object_concept_id and measurement.value_as_concept_id fields, as these fields have multiple acceptable domains. The check can only currently support a single accepted domain
- The withinVisitDates check was previously incorrectly categorized as a Conformance check; it has been recategorized as Plausibility

### New executeDqChecks Parameter

There is now a parameter, `checkSeverity`, which can be used to limit the execution of DQD to `fatal`, `convention`, and/or `characterization` checks. Fatal checks are checks that should never fail, under any circumstance, as they relate to the relational integrity of the CDM. Convention checks are checks on critical OMOP CDM conventions for which failures should be resolved whenever possible; however, some level of failure is unavoidable (i.e., standard concept mapping for source concepts with no suitable standard concept). Characterization checks provide users with an understanding of the quality of the underlying data and generally will need their thresholds modified to match expectations of the source.

### Documentation

- We added 2 more check documentation pages - all DQ checks now have documentation! Check out the newly added pages [here](https://ohdsi.github.io/DataQualityDashboard/articles/checkIndex.html) and please reach out with feedback as we continue improving our documentation!
- We fixed a bug in the exclude checks sample code in CodeToRun.R

DataQualityDashboard 2.6.1
==========================
This release includes:
Expand Down Expand Up @@ -47,7 +68,7 @@ The 3 temporal plausibilty checks are intended to **replace** `plausibleTemporal
For more information on the new checks, please check the [Check Type Definitions](https://ohdsi.github.io/DataQualityDashboard/articles/CheckTypeDescriptions.html) documentation page. If you'd like to disable the deprecated checks, please see the suggested check exclusion workflow in our Getting Started code [here](https://ohdsi.github.io/DataQualityDashboard/articles/DataQualityDashboard.html).

### Check Updates
- The number of measurements checked in `plausibleUnitConceptIds` has been reduced, and the lists of plausible units for those measurements have been re-reviewed and updated for accuracy. This change is intended to improve performance and reliablility of this check. Please file an issue if you would like to contribute additional measurements + plausible units to be checked in the future
- The number of measurements checked in `plausibleUnitConceptIds` has been reduced, and the lists of plausible units for those measurements have been re-reviewed and updated for accuracy. This change is intended to improve performance and reliability of this check. Please file an issue if you would like to contribute additional measurements + plausible units to be checked in the future
- Some erroneous `plausibleValueLow` thresholds have been corrected to prevent false positive failures from occurring

### New Documentation
Expand Down
14 changes: 13 additions & 1 deletion R/executeDqChecks.R
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
#' @param writeToCsv Boolean to indicate if the check results will be written to a csv file. Default is FALSE
#' @param csvFile (OPTIONAL) CSV file to write results
#' @param checkLevels Choose which DQ check levels to execute. Default is all 3 (TABLE, FIELD, CONCEPT)
#' @param checkSeverity Choose which DQ check severity levels to execute. Default is all 3 (fatal, convention, characterization)
#' @param checkNames (OPTIONAL) Choose which check names to execute. Names can be found in inst/csv/OMOP_CDM_v[cdmVersion]_Check_Descriptions.csv. Note that "cdmTable", "cdmField" and "measureValueCompleteness" are always executed.
#' @param cohortDefinitionId The cohort definition id for the cohort you wish to run the DQD on. The package assumes a standard OHDSI cohort table
#' with the fields cohort_definition_id and subject_id.
Expand Down Expand Up @@ -77,6 +78,7 @@
csvFile = "",
checkLevels = c("TABLE", "FIELD", "CONCEPT"),
checkNames = c(),
checkSeverity = c("fatal", "convention", "characterization"),
cohortDefinitionId = c(),
cohortDatabaseSchema = resultsDatabaseSchema,
cohortTableName = "cohort",
Expand Down Expand Up @@ -110,7 +112,16 @@
You passed in ', paste(checkLevels, collapse = ", "))
}

stopifnot(is.null(checkNames) | is.character(checkNames), is.null(tablesToExclude) | is.character(tablesToExclude))
if (!all(checkSeverity %in% c("fatal", "convention", "characterization"))) {
stop('checkSeverity argument must be a subset of c("fatal", "convention", "characterization").
You passed in ', paste(checkSeverity, collapse = ", "))

Check warning on line 117 in R/executeDqChecks.R

View check run for this annotation

Codecov / codecov/patch

R/executeDqChecks.R#L116-L117

Added lines #L116 - L117 were not covered by tests
}

stopifnot(
is.null(checkNames) | is.character(checkNames),
is.character(checkSeverity),
is.null(tablesToExclude) | is.character(tablesToExclude)
)
stopifnot(is.character(cdmVersion))

# Warning if check names for determining NA is missing
Expand Down Expand Up @@ -241,6 +252,7 @@
})]

checkDescriptionsDf <- checkDescriptionsDf[checkDescriptionsDf$checkLevel %in% checkLevels &
checkDescriptionsDf$severity %in% checkSeverity &
checkDescriptionsDf$evaluationFilter != "" &
checkDescriptionsDf$sqlFile != "" &
checkDescriptionsDf$checkName %in% checksToInclude, ]
Expand Down
4 changes: 4 additions & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,10 @@ navbar:
href: articles/checks/withinVisitDates.html
- text: measureConditionEraCompleteness
href: articles/checks/measureConditionEraCompleteness.html
- text: plausibleUnitConceptIds
href: articles/checks/plausibleUnitConceptIds.html
- text: plausibleGenderUseDescendants
href: articles/checks/plausibleGenderUseDescendants.html
hades:
text: hadesLogo
href: https://ohdsi.github.io/Hades
Expand Down
10 changes: 8 additions & 2 deletions docs/404.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 8 additions & 2 deletions docs/LICENSE-text.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 9 additions & 3 deletions docs/articles/AddNewCheck.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 9 additions & 3 deletions docs/articles/CheckStatusDefinitions.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 9 additions & 3 deletions docs/articles/CheckTypeDescriptions.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

16 changes: 13 additions & 3 deletions docs/articles/DataQualityDashboard.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading