backend: allow disabling debug messages #1439
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apage43
force-pushed
the
deverbosify
branch
from
9c6872c to
bb8dddb
Compare
|
obsoleted by upstream |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allows disabling (most) logging in the backend - unless it is enabled by another option explicitly requesting it, or is fatal - and turns this option on for the Python binding builds in CircleCI
Mostly ifdefs around logging statements, except in
llama.cpp(the file) (see submodule change and PR nomic-ai/llama.cpp#6 to update master) where I instead ifdef awayfprintfto avoid introducing more merge conflicts than absolutely necessary.This notably does not prevent the
message that occurs on MacOS - this comes from the Objective-C runtime and is printed because more than one library that is statically linked against llama.cpp has a
GGMLMetalClassdefined in it. This is difficult to prevent because Obj-C classes are not namespaced and cannot be hidden like other symbols in statically linked libraries, and we currently load multiple model backend libraries that may link this code into the same app simultaneously.This is at least harmless, as the class contains no code and is only used to leverage Objective-C functionality on MacOS to find the library path - all of our backend files live in the same directory, so it does not matter which implementation is actually used here. In order to prevent this message from being output we would need to either prevent multiple backend libraries from being resident at once (which would mean making them able to be unloaded and reloaded safely, likely not currently the case), OR somehow generate different classnames for each library that links llama.cpp (a change not likely to be upstreamable as its only needed accommodate loading multiple distinct instances of the llama.cpp library into the same application)