[BUG]: onnx-to-trt fails on conversion of models/phishing-models/phishing-bert-20230517.onnx

[dagardner-nv]

Version

25.02

Which installation method(s) does this occur on?

Source

Describe the bug.

Running the conversion command to convert the OMMX phishing model to TRT as documented in models/README.md results in an error.

Minimum reproducible example

cd models
morpheus --log_level=info tools onnx-to-trt --input_model phishing-models/phishing-bert-20230517.onnx --output_model ./model.plan --batches 1 8 --batches 1 16 --batches 1 32 --seq_length 256 --max_workspace_size 16000

Relevant log output

Click here to see error details

Generating onnx file

Loading ONNX file: '/home/dagardner/work/morpheus/models/phishing-models/phishing-bert-20230517.onnx'

[01/16/2025-16:23:18] [TRT] [W] ModelImporter.cpp:459: Make sure input input_ids has Int64 binding.

[01/16/2025-16:23:18] [TRT] [W] ModelImporter.cpp:459: Make sure input attention_mask has Int64 binding.

Building engine. This may take a while...

[01/16/2025-16:23:18] [TRT] [E] IBuilder::buildSerializedNetwork: Error Code 4: API Usage Error (Dimension mismatch for tensor input_ids and profile 0. At dimension axis 1, profile has min=256, opt=256, max=256 but tensor has 128.)

Writing engine to: {config.output_model}

Traceback (most recent call last):

File "/home/dagardner/work/conda/envs/morpheus/bin/morpheus", line 33, in 

sys.exit(load_entry_point('morpheus', 'console_scripts', 'morpheus')())

File "/home/dagardner/work/morpheus/python/morpheus/morpheus/cli/run.py", line 20, in run_cli

cli(obj={}, auto_envvar_prefix='MORPHEUS', show_default=True, prog_name="morpheus")

File "/home/dagardner/work/conda/envs/morpheus/lib/python3.10/site-packages/click/core.py", line 1161, in call

return self.main(*args, **kwargs)

File "/home/dagardner/work/conda/envs/morpheus/lib/python3.10/site-packages/click/core.py", line 1082, in main

rv = self.invoke(ctx)

File "/home/dagardner/work/conda/envs/morpheus/lib/python3.10/site-packages/click/core.py", line 1697, in invoke

return _process_result(sub_ctx.command.invoke(sub_ctx))

File "/home/dagardner/work/conda/envs/morpheus/lib/python3.10/site-packages/click/core.py", line 1697, in invoke

return _process_result(sub_ctx.command.invoke(sub_ctx))

File "/home/dagardner/work/conda/envs/morpheus/lib/python3.10/site-packages/click/core.py", line 1443, in invoke

return ctx.invoke(self.callback, **ctx.params)

File "/home/dagardner/work/conda/envs/morpheus/lib/python3.10/site-packages/click/core.py", line 788, in invoke

return __callback(*args, **kwargs)

File "/home/dagardner/work/morpheus/python/morpheus/morpheus/cli/utils.py", line 116, in new_func

return f(ctx, *args, **kwargs)

File "/home/dagardner/work/morpheus/python/morpheus/morpheus/cli/commands.py", line 223, in onnx_to_trt

gen_engine(c)

File "/home/dagardner/work/morpheus/python/morpheus/morpheus/utils/onnx_to_trt.py", line 91, in gen_engine

f.write(serialized_engine)

TypeError: a bytes-like object is required, not 'NoneType'

Full env printout

Click here to see environment details

[Paste the results of print_env.sh here, it will be hidden by default]

Other/Misc.

No response

Code of Conduct

• I agree to follow Morpheus' Code of Conduct
• I have searched the open bugs and have found no duplicates for this bug report

[dagardner-nv]

It would appear that the important bit of info is:

[01/16/2025-16:28:22] [TRT] [E] IBuilder::buildSerializedNetwork: Error Code 4: API Usage Error (Dimension mismatch for tensor input_ids and profile 0. At dimension axis 1, profile has min=256, opt=256, max=256 but tensor has 128.)

And changing the --seq_length flag to 128 fixes the issue, will need to double check with DS team.