Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add info for OOM - CGroup Memory configuration #49

Open
mikeswann opened this issue Dec 6, 2024 · 3 comments
Open

Add info for OOM - CGroup Memory configuration #49

mikeswann opened this issue Dec 6, 2024 · 3 comments

Comments

@mikeswann
Copy link

After running into OOM-crashes/quits consistently this patch with swap configured I did some investigating where I found some interesting behaviour, which i believe should be mentioned in the troubleshooting/OOM section of the wiki (or if more consistently observable, possibly introducing a solution via the helper that does this for the game process/service automatically):

Note: i was not able to find any systemd oom-manager on my system, so most likely it is the kernel oom-manager
Swap was not fully utilized, and with z-ram configured not utilized at all.
The source of the oom-killer being invoked seemed to stem from the way memory accounting was being done to define the maximum usable memory, through the CGroup resource-restrictions.
systemctl show <star citizen process slice / service> showed the following:

...
MemoryAvailable=6104674304  #Physical + Swap
EffectiveMemoryMax=33525796864  #Physical
EffectiveMemoryHigh=33525796864  #Physical
...
MemoryHigh=infinity
MemoryMax=infinity
...

In certain conditions, this configuration, even though MemoryHigh and MemoryMax are set to infinity, the oom-killer will get invoked once - EffectiveMemoryMax/High | Physical Memory - is reached.

The solution to override this behaviour was to explicitly set values for MemoryHigh and MemoryMax for the user slice:
systemctl edit user.slice

### Editing /etc/systemd/system/user.slice.d/override.conf
### Anything between here and the comment below will become the contents of the drop-in file

[Slice]
MemoryAccounting=yes #cautionary setting, was reporting as yes before as well
MemoryMax=52G #Physical + swap (or your preference)
MemoryHigh=52G
...

I do not know if there is a way to calculate a value dynamically for these entries. Feel free to edit if so.

After applying the rules with sudo systemctl daemon-reexec and sudo systemctl restart user.slice OR restarting the system, the oom-killer is no longer being invoked and star citizen happily claims memory and swap (observed up to 42G+, where performance becomes questionable but still preferable to a hard-crash)

I am unfortunately not profitient enough in kernel systems to give a more precise answer/reason. If you need more information, feel free to ask.

Noteworthy system information:
Physical Memory: 32GB
Kernel: 6.11.0-9-generic
Distribution: Kubuntu 24.10 x86_64
DE: Plasma 6.1.5 - kwin - wayland
CGroup version: 2

@the-sane
Copy link
Member

the-sane commented Dec 6, 2024

A note from my own system:
My user.slice is configured to set EffectiveMemoryMax/High to 32GB, but inspecting Star Citizen while it's running shows that both are getting set to infinity for the game.

It would seem there's some other piece somewhere making that change and I'd like to know what is doing that. I suspect whatever that is would be the more "proper" place to make the configuration change.

Are you able to get some help from our discord tech support folks to investigate this a bit further?

For reference, I'm on Arch with kernel 6.12.1

@devpatf
Copy link

devpatf commented Dec 12, 2024

same issue here. I'm on ubuntu with kernel 6.8.0-50-generic

@mikeswann
Copy link
Author

same issue here. I'm on ubuntu with kernel 6.8.0-50-generic

Did the fix I mentioned work for you? Feel free to @ me on the discord: thefloppytoast

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants