Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Low-hanging LMDB fruits #15147

Merged
merged 2 commits into from
Feb 13, 2025
Merged

Conversation

miodvallat
Copy link
Contributor

Short description

This simple PR intends to improve the LMDB backend performance in the following situations:

  • upon low-memory conditions, by disabling operating system read ahead on the database file, which could cause some read operations to put database contents in the buffer cache at the expense of memory available to the processes.
  • when processing large RRSET updates, by allocating a temporary data structure once to its desired size, rather than having it grow multiple times to reach its final size.

Checklist

I have:

  • read the CONTRIBUTING.md document
  • compiled this code
  • tested this code
  • included documentation (including possible behaviour changes)
  • documented the code
  • added or modified regression test(s)
  • added or modified unit test(s)

This contributes to not degrading performance too much when the system
memory is almost completely used, by preventing page thrashing due to
the read ahead behaviour of the system.
Copy link
Member

@rgacogne rgacogne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good! Do you know if setting MDB_NORDAHEAD could actually lower performance in some cases (enough RAM for the DB to be loaded in memory)?

@miodvallat
Copy link
Contributor Author

Do you know if setting MDB_NORDAHEAD could actually lower performance in some cases (enough RAM for the DB to be loaded in memory)?

I don't know. This will depend on the workload and the way the database is accessed. Of course, once the whole database is loaded in memory, this doesn't matter anymore 😁

@miodvallat
Copy link
Contributor Author

(also the LMDB documentation does not mention any horrible scenario arising from the use this option)

@miodvallat miodvallat merged commit 6708d17 into PowerDNS:master Feb 13, 2025
83 checks passed
@miodvallat miodvallat deleted the rrset_me_on_fire branch February 13, 2025 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants