Azure CosmosDB Mongo vCore Storage Integrations #16176
Conversation
dosubot
bot
added
the
size:XL
...hat-store-azurecosmosmongovcore/llama_index/storage/chat_store/azurecosmosmongovcore/base.py
Outdated
Show resolved
Hide resolved
…011/llama_index into users/akataria/azureMongoStorage
|
|
||
|
|
||
| class AzureCosmosMongoVCoreKVStore(BaseKVStore, ABC): | ||
| """Creates an AzureCosmosMongoVCoreKVStore.""" |
There was a problem hiding this comment.
Is there a reason that the existing mongo kvstore doesn't work for azure here?
There was a problem hiding this comment.
Seems like its using pymongo under the hood, so I would expect the existing mongo kvstore (and other associated integrations) to work fine 👀
There was a problem hiding this comment.
@logan-markewich , we want to keep our integration separate with a different user agent for tracking purposes.
There was a problem hiding this comment.
@aayush3011 can't we just insert the user-agent in the existing mongodb integrations? Seems like something generally useful for anyone
Otherwise we have completely duplicate code just for a small difference, feels kind of icky
There was a problem hiding this comment.
@logan-markewich we would have to somehow figure out that this is an azure cosmos mongoDB and insert the necessary user agent. To do so, the user then needs to send some keyword like"azure" or "cosmos" as a property when creating using the kvStore.
There was a problem hiding this comment.
@logan-markewich We want to differentiate between whether its a vector store, or any other storage integration. And we have the language in the user agent as well to differentiate between Python and Typescript LlamaIndex implementation
There was a problem hiding this comment.
The kvstore is only used for doctores and index stores -- sorry, but still not seeing the argument for introducing all this code duplication and bloat just to add a user agent
We can have llama-index-docstore-py/llama-index-kvstore-py/llama-index-index-store-py be the user agent for the existing mongodb kvstores for example
There was a problem hiding this comment.
As a maintainer, I do not want to debug two versions of the same class, there's already too much code in the repo as it is 😉
There was a problem hiding this comment.
@logan-markewich Sure, I get your point. Would this be okay if we ask the customer to send "api=azure" in the kwargs and place a check for it to add the azure cosmos db specific user agent. If this approach doesn't seems good, we can go with what you suggested.
There was a problem hiding this comment.
Both seem fine to me, although the approach with the least amount of effort from the user will probably be the easiest
Description
Storage Integration:
New Package?
Did I fill in the
tool.llamahubsection in thepyproject.tomland provide a detailed README.md for my new integration or package?Version Bump?
Did I bump the version in the
pyproject.tomlfile of the package I am updating? (Except for thellama-index-corepackage)Type of Change
Please delete options that are not relevant.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Suggested Checklist:
make format; make lintto appease the lint gods