Multiple rounds of conversations

[tzh5477]

Does Vanna support multiple rounds of dialogue?

Ask again based on the answer to the previous question

[zainhoda]

Not yet -- the simple approach, which is to continue adding user messages to the end of the conversation does not work well in practice because of the types of things that users can potentially ask. They may ask to filter/pivot the table, request changes to the chart, ask for some LLM-based action like extract topics. We have seen that users also somewhat expect that the LLM can "see" the chart (which requires a vision-enabled LLM).

Up until now we were trying to be LLM-agnostic where you could plug in any LLM and it would function roughly similarly. Your request is not uncommon so we will likely be moving to a model where there will be "Basic" functionality that will be possible across any LLM and "Advanced" functionality like what you describe which will only be usable with the top-of-the-line LLMs.

[andreped]

This is a core feature that our team would like as well. I mean, I would like it today even, as we are looking to implement it. At least allow SQL completions to be generated based on chat history. But I can understand that having a generic one-fits-all solution is tricky, as different applications, datasets, and flows require that this might need to be tailored differently.

Would it make sense to make an API in Vanna with one solution, and then developers could override this method, if they want to change the way chat history is handled? Thoughts, @zainhoda? This is already possible to do for table summarization as there is an API for it, hence we can override it if we want, which is something we are looking into trying right now :]

[nightscape]

@andreped did you look into this any further?

[andreped]

@nightscape It is challenging to make a single solution that works well in all scenarios.

What we have tested is simply overriding the question sent to the LLM by providing the entire chat history as the question in the prompt. At least that seems to allow asking questions where it uses the past questions as part of the context.

We have yet to decide on general solution for what to add as answers for past rounds in the chat history. For now we have just excluded that.

[whm233]

In fact, making the web interface like a chat conversation would be fine, otherwise, a new page has to be opened for every question asked

[samuelbolugee]

Is there some way to use Langchain, for example, with Vanna specifically for this?