intro-notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Phylopandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let me introduce you to PhyloPandas. A Pandas dataframe and interface for phylogenetics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import phylopandas as ph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Phylopandas comes with various `read_` methods to load phylogenetic data into a Pandas DataFrame.\n",
    "\n",
    "Check out the various formats by hitting `tab` after `read` in the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ph.read"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try reading some of the sequence files in the `data` folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('data/PF08793_seed.fasta', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ph.read_fasta('data/PF08793_seed.fasta')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ph.read_phylip('data/PF08793_seed.phylip')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ph.read_clustal('data/PF08793_seed.clustal')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Writing data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PhyloPandas attaches a `phylo` accessor to the standard Pandas DataFrame. Inside this accessor are various writing methods, following Pandas syntax, allowing you to write to various sequence formats.\n",
    "\n",
    "To quickly see the writing functions, hit `tab` after `to_` in the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = ph.read_fasta('data/PF08793_seed.fasta')\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.phylo.to_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's write the dataframe back out to fasta. If you don't give a filename, it will return a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s = df.phylo.to_fasta()\n",
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Converting between formats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course, this means you can easily convert between sequence formats. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = ph.read_phylip('data/PF08793_seed.phylip')\n",
    "\n",
    "fasta_str = df.phylo.to_fasta()\n",
    "\n",
    "print(fasta_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading Tree Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Phylopandas can also read in phylogenetic tree data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('data/PF08793_seed.newick', 'r') as f:\n",
    "    print( f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ph.read_newick('data/PF08793_seed.newick')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why is PhyloPandas useful? \n",
    "\n",
    "We already have BioPython, DendroPy, ete3, etc. right?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = ph.read_newick('data/PF08793_seed.newick')\n",
    "\n",
    "df.loc[df.type == \"leaf\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Here is where the real magic happens!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading Sequence *and* Tree Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Phylopandas has the ability to combine sequence and tree data in a single DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read sequences.\n",
    "df = ph.read_fasta('data/PF08793_seed.fasta')\n",
    "\n",
    "# Read tree.\n",
    "df = df.phylo.read_newick('data/PF08793_seed.newick', combine_on='id')\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This enables us to build phylogenetics tools around a single, core dataframe. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Views for PhyloPandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've created a simple, interactive tree viewer powered by Vega. This leverages the \"grammar of phylogenetics\" that PhyloPandas defines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from phylovega import VegaTree\n",
    "\n",
    "# Read data\n",
    "df = ph.read_fasta('data/PF08793_seed.fasta')\n",
    "df = df.phylo.read_newick('data/PF08793_seed.newick', combine_on='id')\n",
    "\n",
    "# Show using VegaTree\n",
    "VegaTree(df).display()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the same DataFrame, we can also show the sequences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import display\n",
    "\n",
    "def Fasta(data=''):\n",
    "    bundle = {}\n",
    "    bundle['application/vnd.fasta.fasta'] = data\n",
    "    bundle['text/plain'] = data\n",
    "    display(bundle, raw=True)\n",
    "\n",
    "Fasta(df.phylo.to_fasta())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Phylogenetics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `phylogenetics` package is our attempt at building tools around PhyloPandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from phylogenetics import PhylogeneticsProject"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Phylogenetics unites many external tools only one single interface and stores their data in a single PhyloPandas DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a working directory\n",
    "working_dir = \"project\"\n",
    "\n",
    "# Initialize a working project.\n",
    "p = PhylogeneticsProject(working_dir, overwrite=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Phylogenetics starts with an alignment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.read_data('data/PF08793_seed.fasta', schema='fasta')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can view that data with the `data` attribute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compute Tree."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Phylogenetics (extremely) simplifies the process of phylogenetics. \n",
    "\n",
    "To compute a tree, simply call `compute_tree`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.compute_tree()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see how that changed the dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is happening under the hood?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each method uses PhyloPandas to prepare the data for an external program, like PhyML or PAML. A subprocess call is made to run the program. Then PhyloPandas is used to read in the results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reconstruct ancestors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To reconstruct ancestors, simply call `compute_reconstruction`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.compute_reconstruction()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, let's see how the DataFrame changed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's just look at ancestors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "anc_df = p.data[p.data.type == 'node']\n",
    "anc_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's write that to file. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fasta_str = anc_df.phylo.to_fasta(sequence_col='ml_sequence')\n",
    "\n",
    "print(fasta_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congratulations! You just reconstructed protein ancestors!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "omnipotent (Python 3.6)",
   "language": "python",
   "name": "omnipotent"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}