examples.md

Examples

Table of Contents

• Using LLMInterface
  • Basic Usage
  • Embeddings
  • Caching
  • Interface Options
  • JSON
• What Can You Do with LLMInterface?
  • Retrieval-Augmented Generation (RAG) using Langchain.js
  • Mixture of Agents (MoA)
  • Miscellaneous

Using LLMInterface

The following examples focus on LLMInterface usage.

Basic Usage

• Chat: Basic LLMInterface.sendMessage() chat usage. This example features an OpenAI compatible structure.
• Prompt: Basic LLMInterface.sendMessage() prompt usage.
• Streaming Mode: LLMInterface.sendMessage() streaming mode prompt usage.
• Set Multiple API Keys: LLMInterface.setApiKey() multiple key usage. This example shows how to set more than one API key at once.

Embeddings

• Embeddings: Basic LLMInterface.embeddings() usage example.
• Embeddings Failover: LLMInterface.embeddings() with default failover usage example.
• Embeddings Custom Failover: LLMInterface.embeddings() with custom failover usage example.

Caching

• Simple Cache: Default file-based cache usage example.
• Memory Cache: High-speed in-memory cache usage example.
• Flat Cache: NPM flat-cache, a JSON flat file cache, usage example. (Node Package)
• Cache Manager: NPM cache-manager, an advanced caching system supporting multiple backends including MongoDB, Memcache, Redis, SQLite, and more, usage example. (Node Package)

Interface Options

• Auto Retry Failed Requests: Controlling retries with interfaceOptions.retryAttempts and interfaceOptions.retryMultiplier usage example.
• Include Original Response: Including the complete original response with interfaceOptions.includeOriginalResponse usage example.
• JSON Repair: Repairing badly formed JSON with interfaceOptions.attemptJsonRepair usage example.

JSON

• JSON Output: Requesting a JSON response using the prompt usage example.
• JSON Repair: Repairing badly formed JSON with interfaceOptions.attemptJsonRepair usage example.
• Native JSON Output: Requesting a JSON response with options.response_format usage example.

What Can You Do with LLMInterface?

The following are some examples using LLMInterface.

Retrieval-Augmented Generation (RAG) using Langchain.js

• Retrieval-Augmented Generation (RAG): Example demonstrating Retrieval-Augmented Generation (RAG) using Langchain.js PromptTemplate and LLMChain.

Mixture of Agents (MoA)

• Mixture of Agents (MoA): Example demonstrating the Mixture of Agents (MoA) concept to improve response quality.

Miscellaneous

• Chart Generation: Example demonstrating Node.js code generation, sandboxed execution, and chart generation.
• RSS Feed Summarization: Example demonstrating RSS feed summarization.