Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] DeleteMany(Expression<Func>) overload causes WAL file to grow uncontrollably if deleting huge amounts of documents #2587

Open
dombrovsky opened this issue Feb 10, 2025 · 0 comments
Labels

Comments

@dombrovsky
Copy link

Version
5.0.17

Describe the bug
Given collection contains millions of documents.
Calling DeleteMany(x => array.Contains(x.Foo)) to delete documents, resulting in million of documents removed, takes around 40 minutes to complete, and causes log file to grow hundreds of gigabytes during this time.

Workaround
If splitting process in smaller batches, e.g. to delete not more than 1000 elements at a time with DeleteMany(x => array.Contains(x.Foo)) call, WAL file size remains small.

Concern
Documents in database where this happens have simple structure and are not exceptionally large (couple of string fields with up to 100 characters each).
While time it takes to delete million documents might make sense, considering million of documents to delete and expression by field that doesn't have index, such aggressive growth of WAL file doesn't seem to have a reasonable explanation. Even if each document deletion caused a new page inserted into WAL, it still won't explain hundreds of gigabytes.
So the concern is that there might be something suboptimal in how WAL is being written in this scenario.

@dombrovsky dombrovsky added the bug label Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant