Handling of context overflow in Local LLMs

[glebashnik]

I confirm that this contribution is made under the terms of the license found in the root directory of this repository's source tree and that I have the authority necessary to make this contribution on behalf of its copyright owner.

This is part of the work on generate indexing expression for document augmentation.
It improves local LLM component, which is a wrapper over llama.cpp for running LLMs locally:

• Adds maxPromptToken config parameter to truncate larger prompts for better control over context size
• Adds handling of context sizes that are larger than configured LLM context size, controlled by contextOverflowPolicy
• Improved error handling when transitioning from async to sync completion method
• Adds blocking wait with timeout for adding requests to executor queue, which prevents immediate error when the queue is full
• Tests that verify content of LLM output, checking for hallucinations because of context overflow and other LLM runtime issues.

[bjorncs]

Consider using @BeforeClass for shared initialization.

[glebashnik]

It is not shared by all test methods.

[bjorncs]

Consider using Duration in the future for time calculation. This have earlier been a typical source for bugs (although not that important here as this is just for logging).