model & transform Probablitites

[StephChev]

Hello,
I'm new to AI and expecially to bertopic.
I'm using Bertopic unsupervised to classify documents into topics. All is working fine. But then we are reusing the model (reloaded as tensors) to find the correct topic for a previous classified document, the correct topic is given but the probabilities are totally differents from the inital run. Is it normal ?

In details:

• BERTopic has been run on many docs and has grouped docA, docB, docC into topicID=5 with probabilitites output (respectvely) of 0.5, 0.42, 0.33.
• model is serialized as tensors
• later model is reloaded, and we do a transform on docA, docB and docC: cluster 5 is given back with proba of 0.92, 0.95, 0.64

Am I wrong somewhere or misunderstanding?

Thnaks a lot for your help
BR
Stephane

[MaartenGr]

All is working fine. But then we are reusing the model (reloaded as tensors) to find the correct topic for a previous classified document, the correct topic is given but the probabilities are totally differents from the inital run. Is it normal ?

Yes, that's normal behavior as the underlying cluster model, HDBSCAN, does an approximation of the probabilities after creating the clusters. Since those probabilities are not an inherent process of clustering, they are bound to differ.

That is assuming you are using either pickle to load the model or directly after you train the model. If you are using safetensors or pytorch, then the underlying cluster model, HDBSCAN, is removed entirely and a new procedure is used for inference, namely cosine similarity between topic and document embeddings. Likewise, these are likely to differ.

[StephChev]

Thnaks a lot for your answer. This is clearer now.
In fact we are building models and saving them using pickle.
However, we needed thame later and so after reloading them we had an issue trying the tranform() as we didn't use "prediction_data=True"...
So we tried to workaround saving them as tensors :) but as we need to have slighly same predictions for same data this was not so helpful.

Just a last question I saw that I could recreate prediction data using : generate_prediction_data()
Could it be a solution to reload pickles bertopic models, call generate_prediction_data(), and save them back ? could this help us to do what wee ned (ie recalculating same or likely same proba for same items) ?

thnaks a lot again for your help
BR
Stephane

[MaartenGr]

Although that might be a solution, it is HDBSCAN-specific I believe and since BERTopic is a modular framework I want to prevent model-specific code as much as possible. You could use that function outside of BERTopic and then save it back to BERTopic if you want but it is not something that I think should be done within BERTopic.