Expectation Suite Store Location

[cosgroveblue]

Suppose we have an expectations_S3_store. Why is is that when we change the location of the folder each time we run great expectations on a data asset, it runs faster? Whereas if we keep this location fixed, the time it takes to run great expectations increases over time?

[austiezr]

Hey @cosgroveblue ! Thanks for reaching out. While we do have eyes on changes to this behavior, currently when running validations, all validations for that asset are re-rendered, meaning that when there are no other validations, the process won't take as long.

Utilizing checkpoints to run your validations should also prevent this re-rendering behavior; are you currently using checkpoints?

Additionally, it is possible to limit the number of validation results rendered by passing a parameter into your Data Docs config; under site_index_builder you can pass validation_results_limit.

For example, something like:

data_docs_sites:
  local_site:
    class_name: SiteBuilder
    show_how_to_buttons: true
    store_backend:
      class_name: TupleFilesystemStoreBackend
      base_directory: uncommitted/data_docs/local_site/
    site_index_builder:
      class_name: DefaultSiteIndexBuilder
      validation_results_limit: 5