Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce Audit Collection size #1257

Open
sggerard opened this issue Jul 16, 2024 · 0 comments
Open

Reduce Audit Collection size #1257

sggerard opened this issue Jul 16, 2024 · 0 comments
Labels

Comments

@sggerard
Copy link
Contributor

sggerard commented Jul 16, 2024

Describe the Bug
The Audit collection in the NRPTI mongodb prod database is huge (19 million records) and has been causing the backups to be slow and even crash.

The Audit collection is used to store events such as record creation and publishing but is also currently storing every search query ever made, and multiple search queries are made by the front end on every page load.

Expected Behaviour
The Audit collection should be a reasonable size and not several million records.

Actual Behaviour
The Audit collection is massive and causing issues with the backups.

Implications
Backups are critical for production data such as NRPTI records and backing up and restoring such a large collection consumes additional resources leading to hitting limits causing crashes.

I would suggest not storing search queries in the Audit table or clearing out records older than say a year.
A migration will likely be necessary.

Affected code:
search.js line 787

QueryUtils.audit(args,
    'Search',
    keywords,
    args.swagger.params.auth_payload ? args.swagger.params.auth_payload
      : { idir_userid: null, displayName: 'public', preferred_username: 'public' }
  );
@sggerard sggerard added the Bug label Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant