How to obtain document-topic probabilities after merging topics in BERTopic?

[ziming00]

Hi Maarten,

I am using the BERTopic library and have trained a model on my documents. After training, I merged several topics using the merge_topics() method. The get_document_info() method provides the topic assignments and the probabilities for each document in the newly merged topics. However, I want to go a step further and calculate the probabilities of each document for all the new topics, not just the assigned one.

Could you please guide me on how to retrieve the full document-topic probability distribution for all topics after performing the merge?

I would greatly appreciate any advice or code snippets that would help me calculate and retrieve the correct document-topic probabilities after the merge operation.

Thanks in advance!

[MaartenGr]

If you have set calculate_probabilities=True, you can run .transform on your docs to get the document-topic probabilities. It will not use HDBSCAN / UMAP for that procedure so it should be quite fast.

[ziming00]

Thank you so much for your helpful response! I tried using .transform() with calculate_probabilities=True, and it worked perfectly, allowing me to obtain the document-topic probabilities as expected. I really appreciate your support and guidance!

Thanks again!