-
Notifications
You must be signed in to change notification settings - Fork 5.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Falkordb vectorstore Integration #16047
Open
A5jadAli
wants to merge
13
commits into
run-llama:main
Choose a base branch
from
A5jadAli:add-falkordb-vectorstore
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+671
−0
Open
Changes from 7 commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
6162fce
Add FalkorDB vector store support
A5jadAli 4a10d7f
tests.py file added in falkordb vectorstore
A5jadAli 4a51aa6
function is inlined
A5jadAli e3056b7
CALL db.indexes() updated
A5jadAli 43dc6c0
requested changes have been made
A5jadAli 6520910
necessary files for integration added
A5jadAli 7952657
docs for falkordb vectorstore added
A5jadAli 194fc4c
falkordb docs file updated
A5jadAli 2cda4ea
update readme
logan-markewich db946e5
linting
logan-markewich c575c11
BUILD
logan-markewich 046c0cf
move to proper folder
logan-markewich d064c47
Fix some issues
logan-markewich File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
29 changes: 29 additions & 0 deletions
29
llama-index-integrations/vector_stores/llama-index-vector-stores-falkordb/.gitignore
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
# Python | ||
__pycache__/ | ||
*.py[cod] | ||
*$py.class | ||
|
||
# Virtual Environment | ||
venv/ | ||
env/ | ||
.env | ||
|
||
# IDE files | ||
.vscode/ | ||
.idea/ | ||
|
||
# Logs | ||
*.log | ||
|
||
# Build files | ||
build/ | ||
dist/ | ||
*.egg-info/ | ||
|
||
# Test files | ||
.pytest_cache/ | ||
.coverage | ||
|
||
# OS generated files | ||
.DS_Store | ||
Thumbs.db |
11 changes: 11 additions & 0 deletions
11
llama-index-integrations/vector_stores/llama-index-vector-stores-falkordb/BUILD
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
load("@rules_python//python:defs.bzl", "py_library") | ||
|
||
py_library( | ||
name = "falkordb", | ||
srcs = ["falkordb.py"], | ||
deps = [ | ||
"//llama_index/core:base", | ||
"//llama_index/core/vector_stores:types", | ||
], | ||
visibility = ["//visibility:public"], | ||
) |
20 changes: 20 additions & 0 deletions
20
llama-index-integrations/vector_stores/llama-index-vector-stores-falkordb/Makefile
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
.PHONY: all format lint test clean | ||
|
||
all: format lint test | ||
|
||
format: | ||
black . | ||
isort . | ||
|
||
lint: | ||
flake8 . | ||
mypy . | ||
|
||
test: | ||
pytest tests/ | ||
|
||
clean: | ||
find . -type f -name "*.pyc" -delete | ||
find . -type d -name "__pycache__" -delete | ||
rm -rf .pytest_cache | ||
rm -rf .mypy_cache |
2 changes: 2 additions & 0 deletions
2
...a-index-integrations/vector_stores/llama-index-vector-stores-falkordb/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
# LlamaIndex Vector_Stores Integration: Falkordb | ||
--- |
141 changes: 141 additions & 0 deletions
141
...s/vector_stores/llama-index-vector-stores-falkordb/falkordb-vectorstore-docs.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,141 @@ | ||
# FalkorDB Vector Store Integration for LlamaIndex | ||
|
||
## Overview | ||
|
||
The FalkorDB Vector Store integration for LlamaIndex allows you to use FalkorDB as a vector store backend for your LlamaIndex applications. This integration supports efficient storage and retrieval of vector embeddings, enabling fast similarity searches and other vector operations. | ||
|
||
## Installation | ||
|
||
To use the FalktorDB vector store with LlamaIndex, you need to install the necessary package: | ||
|
||
```bash | ||
pip install llama-index-vector-stores-falkordb | ||
``` | ||
|
||
## Usage | ||
|
||
### Initializing the FalkorDB Vector Store | ||
|
||
To use the FalkorDB vector store, you first need to initialize it with your FalkorDB connection details: | ||
|
||
```python | ||
from llama_index.vector_stores.falkordb import FalkorDBVectorStore | ||
|
||
vector_store = FalkorDBVectorStore( | ||
url="bolt://localhost:7687", | ||
database="your_database_name", | ||
index_name="your_index_name", | ||
node_label="YourNodeLabel", | ||
embedding_node_property="embedding", | ||
text_node_property="text", | ||
distance_strategy="cosine", | ||
embedding_dimension=1536 | ||
) | ||
``` | ||
|
||
### Adding Documents | ||
|
||
You can add documents to the vector store using the `add` method: | ||
|
||
```python | ||
from llama_index.core.schema import Document | ||
|
||
documents = [ | ||
Document("This is the first document."), | ||
Document("This is the second document.") | ||
] | ||
|
||
vector_store.add(documents) | ||
``` | ||
|
||
### Querying the Vector Store | ||
|
||
To perform a similarity search, use the `query` method: | ||
|
||
```python | ||
from llama_index.core.vector_stores.types import VectorStoreQuery | ||
|
||
query_embedding = [0.1, 0.2, 0.3, ...] # Your query embedding | ||
query = VectorStoreQuery(query_embedding=query_embedding, similarity_top_k=5) | ||
results = vector_store.query(query) | ||
|
||
for node, score in zip(results.nodes, results.similarities): | ||
print(f"Text: {node.text}, Score: {score}") | ||
``` | ||
|
||
## Advanced Features | ||
|
||
### Creating a New Index | ||
|
||
If you need to create a new vector index, you can use the `create_new_index` method: | ||
|
||
```python | ||
vector_store.create_new_index() | ||
``` | ||
|
||
This method will create a new vector index in FalkorDB based on the parameters you provided when initializing the `FalkorDBVectorStore`. | ||
|
||
### Retrieving Existing Index | ||
|
||
To check if an index already exists and retrieve its information: | ||
|
||
```python | ||
exists = vector_store.retrieve_existing_index() | ||
if exists: | ||
print("Index exists with the following properties:") | ||
print(f"Node Label: {vector_store.node_label}") | ||
print(f"Embedding Property: {vector_store.embedding_node_property}") | ||
print(f"Embedding Dimension: {vector_store.embedding_dimension}") | ||
print(f"Distance Strategy: {vector_store.distance_strategy}") | ||
else: | ||
print("Index does not exist") | ||
``` | ||
|
||
### Deleting Documents | ||
|
||
To delete documents from the vector store: | ||
|
||
```python | ||
ref_doc_id = "your_document_id" | ||
vector_store.delete(ref_doc_id) | ||
``` | ||
|
||
### Using Metadata Filters | ||
|
||
You can use metadata filters when querying the vector store: | ||
|
||
```python | ||
from llama_index.core.vector_stores.types import MetadataFilters, MetadataFilter | ||
|
||
filters = MetadataFilters( | ||
filters=[ | ||
MetadataFilter(key="category", value="science", operator=FilterOperator.EQ) | ||
] | ||
) | ||
|
||
query = VectorStoreQuery( | ||
query_embedding=query_embedding, | ||
similarity_top_k=5, | ||
filters=filters | ||
) | ||
|
||
results = vector_store.query(query) | ||
``` | ||
|
||
## Best Practices | ||
|
||
1. **Connection Management**: Ensure that you properly manage your FalkorDB connections, especially in production environments. | ||
2. **Index Naming**: Use descriptive names for your indexes to easily identify them in your FalkorDB instance. | ||
3. **Error Handling**: Implement proper error handling to manage potential issues with connections or queries. | ||
4. **Performance Tuning**: Adjust the `embedding_dimension` and `distance_strategy` parameters based on your specific use case and performance requirements. | ||
|
||
## Troubleshooting | ||
|
||
If you encounter issues: | ||
|
||
1. Check your FalkorDB connection details (URL, database name). | ||
2. Ensure that the FalkorDB server is running and accessible. | ||
3. Verify that the index exists and has the correct properties. | ||
4. Check the FalkorDB logs for any error messages. | ||
|
||
For more information on FalkorDB and its capabilities, refer to the [official FalkorDB documentation](https://falkordb.com/docs/). |
Empty file.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change to falkor://