Matryoshka Representation Learning (MRL) and Binary Quantization (BQ)

MRL is a scaling solution to reduce the dimensionality of the embeddings by truncation. The solution introduces embeddings of various sizes keeping the most important features in earlier dimensions compared to later dimensions. This results in maintaining the most important features while maintaining high quality and cheaper embeddings as used by OpenAI.

Models

• paraphrase-multilingual-MiniLM-L12-v2-sts-2d-matryoshka

pip install torch --index-url https://download.pytorch.org/whl/cu121

pip install -r requirements.txt