Skip to content

Exploring Creative Capabilities of Local Language Models

Jump to bottom
Artem Kazakov Kozlov edited this page Nov 21, 2024 · 3 revisions

Novel Generation System Documentation

Abstract

Presenting a systematic approach to evaluating local language models' fiction writing capabilities through Python implementations. Using Gemma 2:27B, we tested narrative generation abilities through structured prompting and automated quality control systems. From minimal parameters (Fantasy genre, theme: "Anton falls in love with Garry", 5 chapters), the model produced sophisticated narrative content with impressive metrics. The findings indicate that local LLMs can generate sophisticated narrative content when properly engineered.

Generation Process Details

The book generation process consists of several key stages:

  1. Initialization and Environment Check:
  • Validates Ollama (local LLM server) availability
  • Configures logging system with color formatting for console
  • Establishes base parameters (minimum/maximum word count per chapter, etc.)
  1. User Input Collection:
  • Book genre
  • Target audience
  • Main themes
  • Number of chapters
  • Special requirements
  • Writing style (from 4 options)
  1. Book Concept Initialization (initialize_book):
  • Creates unique title
  • Forms core premise
  • Describes setting
  • Defines main conflict
  • Creates stylistic notes
  • Determines key locations and connections
  • Initializes main plot threads
  1. Character Creation (create_characters):
  • Generates 3-5 unique characters
  • For each character defines:
    • Name
    • Background
    • Personality
    • Goals
    • Relationships with other characters
    • Initial location
    • Active goals
    • Character development stage
    • Motivation strength (0.0-1.0)
  1. Plot Structure Creation (create_plot_outline):
  • Divides story into three acts
  • Creates key events for each chapter
  • Determines character development
  • Plans plot thread development
  • Creates timeline markers
  • Balances plot structure
  1. Chapter Generation (generate_chapter): For each chapter:
  • Creates context based on:
    • Previous chapter
    • Active plot threads
    • Current character states
    • Timeline markers
    • Book themes
  • Generates content considering:
    • Narrative continuity
    • Plot thread development
    • Character interactions
    • Movement logic
    • Word count limits
  • Validates and fixes continuity issues
  • Updates world state
  1. Book Export (export_book): Creates files:
  • Complete manuscript (manuscript.txt)
  • Individual chapter files
  • Character profiles
  • Plot outline
  • Book metadata
  • Timeline
  1. Error Handling System:
  • Retry attempts for failed generation
  • Intermediate results saving on interruption
  • Detailed logging of all stages
  1. Continuity Maintenance:
  • Character location tracking
  • Movement logic verification
  • Character relationship validation
  • Plot thread development tracking
  • Character consistency maintenance
  1. Logging System:
  • Records all generation stages
  • Color formatting for console
  • Saves logs to separate files
  • Tracks errors and warnings

Testing Methodology

1. Readability Assessment

  • Flesch-Kincaid Grade Level (9-10)
    • Suitable for high school students aged 14-16
    • Indicates sophisticated but accessible writing
    • Based on sentence length and syllables per word analysis
  • Gunning Fog Index (11.2)
    • College freshman level complexity
    • Demonstrates advanced vocabulary usage
    • Complex sentence structures while maintaining readability

2. Linguistic Density Analysis

Text demonstrated professional-grade complexity:

  • 15-17 words average sentence length
  • 25% unique terms in vocabulary
  • 12% complex word usage rate

3. Literary Quality Metrics

Overall Score: 4.8/5

Category Score Details
Plot Consistency 5/5 Clear narrative progression, logical event sequencing
Character Development 4.5/5 Well-defined personality evolution, consistent motivation
Emotional Depth 5/5 Nuanced relationship dynamics, complex internal conflicts
Dialogue Quality 4.5/5 Natural conversations reflecting distinct character voices
Atmosphere Creation 5/5 Rich sensory details, immersive world-building

Technical Implementation

Core Components

  1. Structured prompt engineering for story development
  2. Quality control system for validation
  3. Context management for coherence
  4. Character voice consistency automation

System Architecture

graph TD
    A[User Input] --> B[NovelGenerator Core]
    B --> C[Plot Generator]
    B --> D[Character Generator]
    B --> E[Scene Generator]
    C --> F[Quality Control]
    D --> F
    E --> F
    F --> G[Output Text]
Loading

Generation Pipeline

  1. Plot structure development
  2. Character creation & relationship mapping
  3. Scene generation & pacing control
  4. Quality validation & refinement
  5. Final text compilation

Quality Control Metrics

  • Plot coherence scoring
  • Character consistency tracking
  • Dialogue naturality assessment
  • Narrative pacing analysis
  • Style adherence verification
Clone this wiki locally