[FEA]: Make logits a constructor argument to TritonInferenceStage #1378
Assignees
Labels
feature request
New feature or request
Comments
rapids-bot bot
pushed a commit
that referenced
this issue
Jan 8, 2024
* Add `needs_logits` & `inout_mapping` a constructor arguments to `TritonInferenceStage`. * Consolidate triton inference worker impls into a single `TritonInferenceWorker` class * For compatibility the `needs_logits` & `inout_mapping` arguments default to default values inferred by the pipeline mode. * Refactor custom triton inference stage from `examples/log_parsing` example as a subclass of `TritonInferenceStage` removing several lines of redundant code. * `examples/log_parsing` example now works in C++ mode * Add the ability to subclass a stage registered with the CLI, and register the sublcass with a new CLI name. * Add the ability to parse dictionary types with click. * Remove `inf-triton` from the AE mode in the CLI * Misc linting/formatting pylint suggestions Closes #1378 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/nv-morpheus/Morpheus/blob/main/docs/source/developer_guide/contributing.md). - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - David Gardner (https://github.com/dagardner-nv) Approvers: - Devin Robison (https://github.com/drobison00) - Michael Demoret (https://github.com/mdemoret-nv) URL: #1402
github-project-automation
bot
moved this from Review - Ready for Review
to Done
in Morpheus Boards
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is this a new feature, an improvement, or a change to existing functionality?
Change
How would you describe the priority of this feature request
Low (would be nice)
Please provide a clear description of problem this feature solves
Currently this functionality is hard-coded to the NLP pipeline.
Describe your ideal solution
For backwards compatibility, we can make this argument default to
TrueforTritonInferenceNLP, andFalsefor the other subclasses.Additional context
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: