Augment the prompt with documentation

[GiacomoPracucci]

If I insert a document into the DB vector, can the prompt also be augmented with a sub-portion of the document (e.g., only two rows are extracted from the document) or will the entire document always be used?

Best Regards

[andreped]

I dont really know this myself. AFAIK, the entire documentation is used in the context if during training you feed the entire documentation as a single string. Perhaps you should chunk it yourself and train vanna on separate chunks. In this case, Vanna should only use a subset of the documentations (sub-portion), similar to question-sql pairs. Is this correct, @zainhoda? I think we ran into the exact same issue right now.

[zainhoda]

Right now it's up to the user to do their own chunking before training on documentation. If we're going to start doing some chunking magic under the hood, I think we'll need to really think about what options we want to expose. This may become less of an issue over time with long-context models but I'd be curious to hear what sort of documentation you're training on @GiacomoPracucci

[GiacomoPracucci]

I uploaded data in tabular format into the documentation. Ideally for me, I would like to be able to extract only those rows that are relevant to the user's question.
I understand that the problem in this example might be at an even earlier step, we would need to be able to read data from a table (currently I have simply structured the doc by columns and rows and made it explicit that it is tabular data).
What I mean is this thing here https://how.wtf/how-to-use-csv-files-in-vector-stores-with-langchain.html (the paragraph "Langchain with CSV data in a vector store". Read the table and then extract only the rows similar to the question).

I replicated the code locally, and if I make a query like, "give me the data related to ...," only the rows related to the requested topic are returned, with the associated columns.

[andreped]

I uploaded data in tabular format into the documentation.

This is where it goes wrong. If you loop across the table and extract each row, you can do vanna.train(documentation=row). This is what we refer to as chunking. In this case, you will get the behaviour you want. If you give the entire table as a single markdown/string or similar, you will not get this behaviour, as this will be interpretated as one gigantic chunk.

Each call to vanna.train(documentation=chunk) stores a new chunk in the vector store which can be retrieved and made available in the context if deemed relevant. This is the exact same behaviour as when storing question-sql pairs.

But I like the idea of this being easier to do, for instance by feeding a pandas.DataFrame to vanna.train(documentation=pandas.DataFrame, it would do the row-wise chunking for us.

[GiacomoPracucci]

Thank you, everything is clearer to me now