What is the best approach to optimize a custom trained NER / SpanCat model ?

[rennanvoa2]

Hello there, I have fine-tuned a couple of custom models using Spacy's and transformer's models and I was wondering if there is a way of optimizing the inference time. I thought that maybe one could parse the model to ONNX and optimize and quantize it, but I searched and found nothing about it.

Is there a way of applying optimization and quantization to Spacy models?

[polm]

We have had a few questions about ONNX before - see #7704 or here - but currently we don't have support for that or any reports of anyone doing it successfully.

If you haven't seen it yet, the speed FAQ may be helpful.

[polm]

Also, looks like you found it before, but #10006 is related.