chatter

adapter over LLMs interface

sub-module	doc	about
		chatter types
		AWS Bedrock LLMs
		AWS Bedrock Batch Inference
		LLMs query/reply caching
		OpenAI LLMs

---

The library is adapter over various popular Large Language Models (LLMs) tuned for text generation: AWS BedRock, OpenAI.

Inspiration

A good prompt has 4 key elements: Role, Task, Requirements, Instructions. "Are You AI Ready? Investigating AI Tools in Higher Education – Student Guide"

In the research community, there was an attempt for making standardized taxonomy of prompts for large language models (LLMs) to solve complex tasks. It encourages the community to adopt the TELeR taxonomy to achieve meaningful comparisons among LLMs, facilitating more accurate conclusions and helping the community achieve consensus on state-of-the-art LLM performance more efficiently.

The library addresses the LLMs comparisons by

• Creating generic trait to "interact" with LLMs;
• Enabling prompt definition into seven distinct levels;
• Supporting variety of LLMs.

type Chatter interface {
	Prompt(context.Context, []fmt.Stringer, ...func(*Options)) (string, error)
}

Getting started

The latest version of the library is available at main branch of this repository. All development, including new features and bug fixes, take place on the main branch using forking and pull requests as described in contribution guidelines. The stable version is available via Golang modules.

package main

import (
	"context"
	"fmt"

	"github.com/kshard/chatter"
	"github.com/kshard/chatter/bedrock"
)

func main() {
	assistant, err := bedrock.New(
		bedrock.WithLLM(bedrock.LLAMA3_0_8B_INSTRUCT),
	)
	if err != nil {
		panic(err)
	}

	var prompt chatter.Prompt
	prompt.WithTask("Extract keywords from the text: %s", /* ... */)

	reply, err := assistant.Prompt(context.Background(), prompt.ToSeq())
	if err != nil {
		panic(err)
	}

	fmt.Printf("==> (%d)\n%s\n", assistant.ConsumedTokens(), reply)
}

Prompt

Package chatter provides utilities for creating and managing structured prompts for language models.

The Prompt type allows you to create a structured prompt with various sections such as task, rules, feedback, examples, context, and input. This helps in maintaining semi-structured prompts while enabling efficient serialization into textual prompts.

{task}. {guidelines}.
1. {requirements}
2. {requirements}
3. ...
{feedback}
{examples}
{context}
{context}
...
{input}

Example usage:

var prompt chatter.Prompt{}

prompt.WithTask("Translate the following text")

// Creates a guide section with the given note and text.
// It is complementary paragraph to the task.
prompt.With(chatter.Guide("Please translate the text accurately"))

// Creates a rules / requirements section with the given note and text.
prompt.With(chatter.Rules(
  "Strictly adhere to the following requirements when generating a response.",
  "Do not use any slang or informal language",
  "Do not invent new, unkown words",
))

// Creates a feedback section with the given note and text.
prompt.With(chatter.Feedback(
  "Improve the response based on feedback",
  "Previous translations were too literal.",
))

// Create example of input and expected output.
prompt.With(chatter.Example{
  Input: `["Hello"]`,
  Reply: `["Hola"]`
})

// Creates a context section with the given note and text.
prompt.With(chatter.Context(
  "Below are additional context relevant to your goal task.",
  "The text is a formal letter",
))

// Creates an input section with the given note and text.
prompt.With(chatter.Input(
  "Translate the following sentence",
  "Hello, how are you?",
))

How To Contribute

The library is MIT licensed and accepts contributions via GitHub pull requests:

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Added some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request

The build and testing process requires Go version 1.21 or later.

build and test library.

git clone https://github.com/kshard/chatter
cd chatter
go test ./...

commit message

The commit message helps us to write a good release note, speed-up review process. The message should address two question what changed and why. The project follows the template defined by chapter Contributing to a Project of Git book.

bugs

If you experience any issues with the library, please let us know via GitHub issues. We appreciate detailed and accurate reports that help us to identity and replicate the issue.

License