llm-finetuning-notes

Course Notes from LLM Fine-Tuning for Data Scientists and Software Engineers.

Resources

AI Product Evaluation

• Your AI Product Needs Evals (hamel.dev)
• Langtrace AI - Monitor, eval & improve your LLM apps (langtrace.ai)
• Observability for LLMs (honeycomb.io)
• Inspect, a framework for large language model evaluations created by the UK AI Safety Institute (ukgovernmentbeis.github.io)

Programming and Development Tools

• DSPy: Programming—not prompting—Foundation Models (github.com)
• Cohere Toolkit to quickly build and deploy RAG apps (cohere.com)
• Open UI (v0.dev) and Open UI GitHub (github.com)
• Effortless Python web applications (shiny.posit.co)
• Fireworks’ GPT-4-level function calling model (fireworks.ai)
• Code for the Hermes Pro Large Language Model to perform function calling based on the provided schema (github.com)
• InternVL - A Pioneering Open-Source Alternative to GPT-4V (github.com)
• Gorilla: Large Language Model Connected with Massive APIs (gorilla.cs.berkeley.edu)
• Notebook fine-tuning on a Captcha image dataset (github.com)
• Get your LLM app from prototype to production (langchain.com)

Tokenization and Fine-Tuning

• Tokenization Gotchas (hamel.dev)
• Fine-tuning: Axolotl vs Unsloth vs TorchTune (swaroopch.com)
• Curating LLM data (hamel.dev)
• Tools for curating LLM data (hamel.dev)
• Notebook fine-tuning on a Captcha image dataset (github.com)

Prompt Engineering

• Anthropic's Prompt Engineering Interactive Tutorial (docs.google.com)
• Fuck You, Show Me The Prompt (hamel.dev)
• Series: Prompt injection (simonwillison.net)

Research Papers and Studies

• Constitutional AI: Harmlessness from AI Feedback (arxiv.org)
• The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions (arxiv.org)
• Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study (arxiv.org)
• RAFT: Adapting Language Model to Domain Specific RAG (arxiv.org)
• Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? (arxiv.org)
• Your Language Model is Secretly a Reward Model (arxiv.org)
• The Unreasonable Ineffectiveness of the Deeper Layers (twitter.com)

News and Articles

• Latent Space The AI Engineer newsletter (latent.space)
• Air Canada Has to Honor a Refund Policy Its Chatbot Made Up (wired.com)

Educational Content

• YT video explaining transformers (youtube.com)
• But what is a GPT? Visual intro to transformers | Chapter 5, Deep Learning (youtube.com)