Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Docs] Point to smoothquant guide after failure to match #1074

Closed
wants to merge 1 commit into from

Conversation

kylesayrs
Copy link
Collaborator

@kylesayrs kylesayrs commented Jan 15, 2025

Purpose

  • Encourage open source community engagement by pointing to resources for users to add their own smoothquant mappings

Related issues

Changes

  • Catch errors that result from failure to match mappings to modules. Have that error point users towards the smoothquant guide
  • Reduce error verbosity when match_layers_params fails
    • While there's some sense in printing the whole module, the module repr is almost always not unique and is not any more helpful than just printing the name of the module

Testing

  • TODO: Test and show resulting UI

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Copy link

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

@kylesayrs
Copy link
Collaborator Author

This has already been implemented by the handle_mapping_resolution_errors wrapper

RuntimeError: Error resolving mappings for given architecture.Please refer to the README at /home/ksayers/llm-compressor/src/llmcompressor/modifiers/smoothquant/README.md for more information.

@kylesayrs kylesayrs closed this Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant