notebook/yohane.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lACVssChmYNf"
      },
      "source": [
        "# yohane <img src=\"https://hikari.butaishoujo.moe/p/06bfbaf3/680954239740411973.png\" height=\"24px\" width=\"24px\" style=\"display:inline;object-fit:contain;vertical-align:middle\" >\n",
        "\n",
        "---\n",
        "\n",
        "Please click the badge below to open the latest version of the notebook:\n",
        "\n",
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/Japan7/yohane/blob/main/notebook/yohane.ipynb\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
        "</a>\n",
        "\n",
        "---\n",
        "\n",
        "**Before proceeding, change your runtime type to GPU (Toolbar > Runtime > Change runtime type > T4 GPU).**\n",
        "\n",
        "![](https://hikari.butaishoujo.moe/p/bde50ce2/out.png)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xwJTVAXcu-Ln"
      },
      "source": [
        "## Install\n",
        "\n",
        "Execute the next cells to install yohane.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "WreI26vRmKmO"
      },
      "outputs": [],
      "source": [
        "!python3 --version\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "DUuXDO8enIK2"
      },
      "outputs": [],
      "source": [
        "%%bash\n",
        "REPO_URL=https://github.com/Japan7/yohane.git\n",
        "LATEST_TAG=$(git ls-remote --tags --sort -v:refname $REPO_URL | head -n1 | cut --delimiter='/' --fields=3)\n",
        "\n",
        "pip3 install git+$REPO_URL@$LATEST_TAG\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Restart the runtime if prompted."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "!pip3 show yohane\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uHDmLc9R7x8J"
      },
      "source": [
        "## Parameters\n",
        "\n",
        "The next cells will set the parameters for the yohane pipeline."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PGHogUA-7qcz"
      },
      "outputs": [],
      "source": [
        "# @title Song { display-mode: \"form\" }\n",
        "# @markdown Run this cell and use the form below to **upload your song**.\n",
        "#\n",
        "# @markdown It can be either an audio or video file.\n",
        "#\n",
        "# @markdown **If it fails, try another browser or upload your file manually in the Files section of the left side bar.**\n",
        "\n",
        "from google.colab import files\n",
        "\n",
        "files.upload_file(\"song\")\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "luFhCjhb-Dfm"
      },
      "outputs": [],
      "source": [
        "# @title Lyrics { display-mode: \"form\", run: \"auto\" }\n",
        "# @markdown Run this cell and **paste your lyrics** in the box below.\n",
        "\n",
        "from IPython.display import display\n",
        "from ipywidgets import Layout, Textarea\n",
        "\n",
        "lyrics_area = Textarea(layout=Layout(width=\"100%\", height=\"200px\"))\n",
        "display(lyrics_area)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "U0i4FT5V-NSG"
      },
      "outputs": [],
      "source": [
        "# @title Vocals Extractor { display-mode: \"form\", run: \"auto\" }\n",
        "# @markdown Run this cell and select the desired **Vocals Extractor**:\n",
        "# @markdown - `VocalRemoverVocalsExtractor` is based on the [`vocal-remover`](https://github.com/tsurumeso/vocal-remover) library. Take this one if you don't know what to choose.\n",
        "# @markdown - `HybridDemucsVocalsExtractor` uses `torchaudio`'s [Hybrid Demucs model](https://pytorch.org/audio/2.1.0/tutorials/hybrid_demucs_tutorial.html) which is faster but less aggressive.\n",
        "# @markdown - `None` if you don't care and want to skip the vocals extraction step.\n",
        "\n",
        "from yohane.audio import VocalRemoverVocalsExtractor, HybridDemucsVocalsExtractor\n",
        "\n",
        "vocals_extractor_class = VocalRemoverVocalsExtractor # @param [\"VocalRemoverVocalsExtractor\", \"HybridDemucsVocalsExtractor\", \"None\"] {type:\"raw\"}\n",
        "vocals_extractor = vocals_extractor_class() if vocals_extractor_class is not None else None\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lW-jK163FcBW"
      },
      "source": [
        "## Run\n",
        "\n",
        "When ready, execute the next cells to run the pipeline.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nGnkPuSzFZzk"
      },
      "outputs": [],
      "source": [
        "# @title Generate\n",
        "# @markdown **Replace the song filename here if you uploaded it manually**\n",
        "\n",
        "import logging\n",
        "from pathlib import Path\n",
        "from yohane import Yohane\n",
        "\n",
        "logging.basicConfig(level=\"INFO\", force=True)\n",
        "\n",
        "song_filename = \"song\" # @param {type:\"string\"}\n",
        "\n",
        "yohane = Yohane(vocals_extractor)\n",
        "yohane.load_song(Path(song_filename))\n",
        "yohane.load_lyrics(lyrics_area.value)\n",
        "yohane.extract_vocals()\n",
        "yohane.force_align()\n",
        "subs = yohane.make_subs()\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bgfd-NS6INkN"
      },
      "outputs": [],
      "source": [
        "# @title Save and download\n",
        "\n",
        "from google.colab import files\n",
        "\n",
        "subs.save(\"karaoke.ass\")\n",
        "files.download(\"karaoke.ass\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YVB0lS8ENJvo"
      },
      "source": [
        "The karaoke should have been downloaded. If not, open Files in the left side bar and look for `karaoke.ass`.\n",
        "\n",
        "**Next recommended steps in Aegisub:**\n",
        "\n",
        "1. Load the .ass and the video\n",
        "2. Replace the _Default_ style with your own\n",
        "3. Due to the normalization during the process, lines are lowercased and special characters have been removed: use the original lines in comments to fix the timed lines\n",
        "4. Subtitle > Select Lines… > check _Comments_ and _Set selection_ > OK and delete the selected lines\n",
        "5. Listen to each line and fix their End time\n",
        "6. Iterate over each line in karaoke mode and merge/fix syllable timings\n",
        "\n",
        "**Happy editing!**\n",
        "\n",
        "![](https://hikari.butaishoujo.moe/p/bd9b7a37/genjitsunoyohane-ep1-scr01.jpg)\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.11.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}