4. Whisper.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0204487c",
   "metadata": {},
   "source": [
    "<div style=\"width: 100%; overflow: hidden;\">\n",
    "    <div style=\"width: 150px; float: left;\"> <img src=\"https://raw.githubusercontent.com/DataForScience/Networks/master/data/D4Sci_logo_ball.png\" alt=\"Data For Science, Inc\" align=\"left\" border=\"0\" width=150px> </div>\n",
    "    <div style=\"float: left; margin-left: 10px;\"> <h1>LLMs for Data Science</h1>\n",
    "    <h1>Text to Speech with OpenAI</h1>\n",
    "        <p>Bruno Gonçalves<br/>\n",
    "        <a href=\"http://www.data4sci.com/\">www.data4sci.com</a><br/>\n",
    "            @bgoncalves, @data4sci</p></div>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e355f937",
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter, defaultdict\n",
    "import random\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt \n",
    "\n",
    "import openai\n",
    "from openai import OpenAI\n",
    "\n",
    "import tqdm as tq\n",
    "from tqdm.notebook import tqdm\n",
    "\n",
    "import watermark\n",
    "\n",
    "%load_ext watermark\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a285801f",
   "metadata": {},
   "source": [
    "We start by printing out the versions of the libraries we're using for future reference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8c88a59c",
   "metadata": {},
   "outputs": [],
   "source": [
    "%watermark -n -v -m -g -iv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47155b58",
   "metadata": {},
   "source": [
    "Load default figure style"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "24195532",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.style.use('d4sci.mplstyle')\n",
    "colors = plt.rcParams['axes.prop_cycle'].by_key()['color']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94486e90",
   "metadata": {},
   "source": [
    "# Audio to Text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "03462fff",
   "metadata": {},
   "outputs": [],
   "source": [
    "client = OpenAI()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03e6a05b",
   "metadata": {},
   "source": [
    "Let us parse a small local file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "14ef820a",
   "metadata": {},
   "outputs": [],
   "source": [
    "!open data/gettysburg10.wav"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "20b90dae",
   "metadata": {},
   "outputs": [],
   "source": [
    "transcript = client.audio.transcriptions.create(\n",
    "    file = open(\"data/gettysburg10.wav\", \"rb\"),\n",
    "    model = \"whisper-1\",\n",
    "    response_format=\"text\",\n",
    "    language=\"en\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0d2e533",
   "metadata": {},
   "source": [
    "And the transcript is simply:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b8114f29",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(transcript)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc9cc136",
   "metadata": {},
   "source": [
    "We can also ask for SRT formatted output, that includes time indices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "929d4de4",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "transcript = client.audio.transcriptions.create(\n",
    "    file = open(\"data/gettysburg10.wav\", \"rb\"),\n",
    "    model = \"whisper-1\",\n",
    "    response_format=\"srt\",\n",
    "    language=\"en\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d3ce2020",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(transcript)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c92d12a",
   "metadata": {},
   "source": [
    "And ask it to translate the text directly into Spanish"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "56c6b7bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "transcript = client.audio.transcriptions.create(\n",
    "    file = open(\"data/gettysburg10.wav\", \"rb\"),\n",
    "    model = \"whisper-1\",\n",
    "    response_format=\"text\",\n",
    "    language=\"es\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "0ae2d5ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(transcript)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "509db3bb",
   "metadata": {},
   "source": [
    "# Text to Speech"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0aad4ad8",
   "metadata": {},
   "source": [
    "Now the opposite approach, going from written text to high quality audio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "98069bb4",
   "metadata": {},
   "outputs": [],
   "source": [
    "quote = \"\"\"\n",
    "Scientists have calculated that the chances of something so patently absurd \n",
    "actually existing are millions to one.\n",
    "But magicians have calculated that million-to-one chances crop up nine times out of ten.\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3601c5db",
   "metadata": {},
   "source": [
    "You can learn more about text to speech (and sample the various voices) in the [Official documentation](https://platform.openai.com/docs/guides/text-to-speech/quickstart)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "51ebbe89",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "audio = client.audio.speech.create(\n",
    "    input=quote, \n",
    "    model=\"tts-1\", \n",
    "    voice='fable',\n",
    "    response_format='mp3')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "250e740c",
   "metadata": {},
   "source": [
    "Which we can write directly to a file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "0d897c4f",
   "metadata": {},
   "outputs": [],
   "source": [
    "audio.write_to_file('data/pratchett.mp3')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "15d6eef7",
   "metadata": {},
   "outputs": [],
   "source": [
    "!open data/pratchett.mp3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16006f9a",
   "metadata": {},
   "source": [
    "<center>\n",
    "     <img src=\"https://raw.githubusercontent.com/DataForScience/Networks/master/data/D4Sci_logo_full.png\" alt=\"Data For Science, Inc\" align=\"center\" border=\"0\" width=300px> \n",
    "</center>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}