BF: restrict value of get_fdmax #417
Open
+27
−8
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In my case I kept finding celery running at 100% and doing nothing. py-spy
pointed to the close_open_fds and then ulimit inside the container showed gory
detail of
❯ docker run -it --rm --entrypoint bash dandiarchive/dandiarchive-api -c "ulimit -n"
1073741816
situation is not unique to me. See more at
dandi/dandi-cli#1488
yarikoptic
added a commit
to dandi/dandi-cli
that referenced
this pull request
Nov 20, 2024
yarikoptic
added a commit
to dandi/dandi-cli
that referenced
this pull request
Nov 20, 2024
yarikoptic
added a commit
to dandi/dandi-cli
that referenced
this pull request
Nov 20, 2024
|
hm, why pre-commit.ci is even configured if there is no |
auvipy
requested changes
Mar 5, 2025
|
I would be happy to elaborate! ATM I can only reiterate what tried to describe in original description -- on some systems ulimit would return HUGE number for maximal number of open descriptiors, which would be infeasible to loop through. So, billiard should not try to loop through all the possible billion of them. |
|
May be we can add some unit tests for the suggested changes as well |
|
sorry -- too busy ATM, feel welcome to contribute some by directly pushing to my branch for this PR, I did
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In my case I kept finding celery running at 100% and doing nothing. py-spy pointed to the close_open_fds and then ulimit inside the container showed gory detail of
situation is not unique to me. See more at
I verified that with this fix my celery container gets unstuck and proceeds to report useful errors ;)