Modern Python Idioms.ipynb

{
 "metadata": {
  "name": "Modern Python Idioms"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Context Managers\n",
      "\n",
      "A _context manager_ is a way to specify particular runtime contexts in your python code.\n",
      "\n",
      "Most used example is file opening"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "with open(\"example.json\") as file_handle:\n",
      "    data = file_handle.read()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 2
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In the _context_ of the `with` block there is an open file object `file_handle`. Since having a file open takes limited system resources, it's always important to close open files."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "file_handle = open(\"example.json\")\n",
      "data = file_handle.read()\n",
      "file_handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 9
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "One should however always do"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "file_handle = open(\"example.json\")\n",
      "try:\n",
      "    data = file_handle.read()\n",
      "except IOError:\n",
      "    pass\n",
      "except Exception:\n",
      "    print(\"Got exception\")\n",
      "else:\n",
      "    print(\"Got no excpetion\")\n",
      "finally:\n",
      "    file_handle.close()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "In the case using the managed context, parts of setting up things, and ensuring that things get properly torn down, are handled automatically when _entering_ and _exiting_ the context.\n",
      "\n",
      "This pattern is extremely common, therefore we have the `with` statement which uses what we call context managers to create contexts.\n",
      "\n",
      "A context manager is a class whose objects (called a _context guard_) has an `__enter__` method and an `__exit__` method "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class ContextManager(object):\n",
      "    def __enter__(self):\n",
      "        pass\n",
      "    \n",
      "    def __exit__(self, type, value, traceback):\n",
      "        pass"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 13
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "When using the `with` statement what actually happens is (equivalent to)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "context_guard = ContextManager()\n",
      "value = context_guard.__enter__()\n",
      "exc = True\n",
      "try:\n",
      "    try:\n",
      "        var = value\n",
      "        pass  # Here the code in the context\n",
      "              # would go.\n",
      "\n",
      "    except:\n",
      "        exc = False\n",
      "        if not context_guard.__exit__(*sys.exc_info()):\n",
      "            raise\n",
      "\n",
      "finally:\n",
      "    if exc:\n",
      "        context_guard.__exit__(None, None, None)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 20
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "If `__enter__` returns a value, this can be caught by `... as var`\n",
      "\n",
      "One can suppress exceptions under certain conditions with the return value of the `__exit__` method\n",
      "\n",
      "Example: "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import sys\n",
      "from StringIO import StringIO\n",
      "\n",
      "class redirect_stdout:\n",
      "     def __init__(self, target):\n",
      "         self.stdout = sys.stdout\n",
      "         self.target = target\n",
      "\n",
      "     def __enter__(self):\n",
      "         sys.stdout = self.target\n",
      "\n",
      "     def __exit__(self, type, value, tb):\n",
      "         sys.stdout = self.stdout"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 1
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "out = StringIO()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 2
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "with redirect_stdout(out):\n",
      "    print(\"Prints to sys.stdout\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 3
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "print(\"This too\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "This too\n"
       ]
      }
     ],
     "prompt_number": 4
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "out.getvalue()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 5,
       "text": [
        "'Prints to sys.stdout\\n'"
       ]
      }
     ],
     "prompt_number": 5
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Decorators\n",
      "\n",
      "Say I have some functions defined in a module, among them these:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def my_add(x, y):\n",
      "    return x + y\n",
      "\n",
      "def my_subtract(x, y):\n",
      "    return x - y\n",
      "\n",
      "def my_multiply(x, y):\n",
      "    return x * y"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 7
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "But I decide that whenever I'm calling this subset of functions, I want to print the parameters.\n",
      "That means I would have to modify each function this way:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def my_add(x, y):\n",
      "    print(x, y)\n",
      "    return x + y\n",
      "\n",
      "my_add(6,7)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "(6, 7)\n"
       ]
      },
      {
       "output_type": "pyout",
       "prompt_number": 8,
       "text": [
        "13"
       ]
      }
     ],
     "prompt_number": 8
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The printing functionality implementation would be the same for every function. This violates the principle of DRY: Don't Repeat Yourself.\n",
      "\n",
      "A better way would be to make the implementation of the argument printing once, and apply that implementation to each function."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def print_args(function):\n",
      "    def wrapper(*args, **kwargs):\n",
      "        print(\"args: {}\".format(args))\n",
      "        print(\"kwargs: {}\".format(kwargs))\n",
      "\n",
      "        return function(*args, **kwargs)\n",
      "    \n",
      "    return wrapper"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 9
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "def my_add(x, y):\n",
      "    return x + y\n",
      "\n",
      "my_add = print_args(my_add)\n",
      "\n",
      "my_add(3,4)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "args: (3, 4)\n",
        "kwargs: {}\n"
       ]
      },
      {
       "output_type": "pyout",
       "prompt_number": 10,
       "text": [
        "7"
       ]
      }
     ],
     "prompt_number": 10
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "There is a python shorthand for the definition and modification, which is a **decorator**."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "@print_args\n",
      "def my_subtract(x, y):\n",
      "    return x - y\n",
      "\n",
      "my_subtract(5,2)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "args: (5, 2)\n",
        "kwargs: {}\n"
       ]
      },
      {
       "output_type": "pyout",
       "prompt_number": 11,
       "text": [
        "3"
       ]
      }
     ],
     "prompt_number": 11
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "my_add(1,2,3,5, a1=1, a2=4)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "TypeError",
       "evalue": "my_add() takes exactly 2 arguments (6 given)",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
        "\u001b[0;32m<ipython-input-12-17c0c25df71d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mmy_add\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m3\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ma1\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ma2\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m4\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
        "\u001b[0;32m<ipython-input-9-a42d230fb22e>\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m      4\u001b[0m         \u001b[0;32mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"kwargs: {}\"\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mfunction\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mwrapper\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
        "\u001b[0;31mTypeError\u001b[0m: my_add() takes exactly 2 arguments (6 given)"
       ]
      },
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "args: (1, 2, 3, 5)\n",
        "kwargs: {'a1': 1, 'a2': 4}\n"
       ]
      }
     ],
     "prompt_number": 12
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "There are some built-in decorators in python, and many packages uses them for functionality.\n",
      "\n",
      "For example, in our production pipeline we use a `task` decorator from the package Celery to define available remotely executable \"tasks\" \n",
      "\n",
      "( https://github.com/SciLifeLab/bcbb/blob/master/nextgen/bcbio/distributed/tasks.py#L84 )\n",
      "\n",
      "    ...\n",
      "    \n",
      "    from celery.task import task\n",
      "    from bcbio.pipeline import lane\n",
      "\n",
      "    ...\n",
      "\n",
      "    @task\n",
      "    def remove_contaminants(*args):\n",
      "        return lane.remove_contaminants(*args)\n",
      "\n",
      "    ...\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### `@property` decorator\n",
      "\n",
      "You used to have a simple property of a class, but years later you realized this property would be more appropriate as returned value. But you don't want to break the functionality of the class for all the people who are already using your package."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class CLS(object):\n",
      "    def __init__(self):\n",
      "        self.a = 1"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 13
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "obj = CLS()\n",
      "obj.a"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 14,
       "text": [
        "1"
       ]
      }
     ],
     "prompt_number": 14
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class CLS(object):\n",
      "    def __init__(self):\n",
      "        self.b = 3.\n",
      "        self.c = 2.\n",
      "    \n",
      "    @property\n",
      "    def a(self):\n",
      "        return self.b / self.c"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 15
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "obj = CLS()\n",
      "\n",
      "obj.a"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 16,
       "text": [
        "1.5"
       ]
      }
     ],
     "prompt_number": 16
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "obj.b = 4.\n",
      "\n",
      "obj.a"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 18,
       "text": [
        "2.0"
       ]
      }
     ],
     "prompt_number": 18
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Property objects have `getter`, `setter` and `deleter` attributes which can be used as decorators."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class CLS(object):\n",
      "    def __init__(self):\n",
      "        self.b = 3.\n",
      "        self.c = 2.\n",
      "    \n",
      "    @property\n",
      "    def a(self):\n",
      "        return self.b / self.c\n",
      "    \n",
      "    @a.setter\n",
      "    def a(self, value):\n",
      "        self.c = 1.0\n",
      "        self.b = value"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 19
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "obj = CLS()\n",
      "\n",
      "print(\"Initial b, c, a: \\n{}, {}, {}\".format(obj.b, obj.c, obj.a))\n",
      "\n",
      "obj.a = 5.0\n",
      "\n",
      "print(\"\")\n",
      "print(\"b, c, a after a being set: \\n{}, {}, {}\".format(obj.b, obj.c, obj.a))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "Initial b, c, a: \n",
        "3.0, 2.0, 1.5\n",
        "\n",
        "b, c, a after a being set: \n",
        "5.0, 1.0, 5.0\n"
       ]
      }
     ],
     "prompt_number": 20
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Class Decorators\n",
      "\n",
      "One can also define decorators that take a class and return a new class. E.g. to add some pattern of methods or standard fields. This can be a quicker (to implement) alternative to MetaClasses."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Object Orientation\n",
      "\n",
      "(Remember inheritance? See http://software-carpentry.org/4_0/oop/inherit.html if you need a refresher)"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class A(object):\n",
      "    a = []\n",
      "\n",
      "class B(A):\n",
      "    def __init__(self, name):\n",
      "        self.name = name\n",
      "        self.a.append(self.name)\n",
      "    \n",
      "    def print_list(self):\n",
      "        print(self.a)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 21
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b1 = B(\"1\")\n",
      "b2 = B(\"2\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 22
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b1.print_list()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['1', '2']\n"
       ]
      }
     ],
     "prompt_number": 23
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b2.a is b1.a"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 25,
       "text": [
        "True"
       ]
      }
     ],
     "prompt_number": 25
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b3 = B(\"3\")\n",
      "b1.print_list()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['1', '2', '3']\n"
       ]
      }
     ],
     "prompt_number": 26
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "A.a"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 27,
       "text": [
        "['1', '2', '3']"
       ]
      }
     ],
     "prompt_number": 27
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "del b3"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 28
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b1.print_list()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['1', '2', '3']\n"
       ]
      }
     ],
     "prompt_number": 29
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b3"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "ename": "NameError",
       "evalue": "name 'b3' is not defined",
       "output_type": "pyerr",
       "traceback": [
        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m\n\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
        "\u001b[0;32m<ipython-input-30-c1b703401214>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mb3\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
        "\u001b[0;31mNameError\u001b[0m: name 'b3' is not defined"
       ]
      }
     ],
     "prompt_number": 30
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class A(object):\n",
      "    a = []\n",
      "\n",
      "\n",
      "class B(A, object):\n",
      "    def __init__(self, name):\n",
      "        self.name = name\n",
      "        self.a.append(self.name)\n",
      "    \n",
      "    def print_list(self):\n",
      "        print(self.a)\n",
      "        \n",
      "    def __del__(self):\n",
      "        self.a.remove(self.name)\n",
      "\n",
      "\n",
      "b1 = B(\"1\")\n",
      "b2 = B(\"2\")\n",
      "b3 = B(\"3\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 31
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b1.print_list()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['1', '2', '3']\n"
       ]
      }
     ],
     "prompt_number": 32
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "del b3"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 33
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b1.print_list()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['1', '2']\n"
       ]
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Python has many method names starting and ending with `__` which makes classes integrate more with the standard Python syntax.\n",
      "\n",
      "These are normally called \"Magic methods\" or \"Dunder methods\".\n",
      "\n",
      "This way one can make class objects behave in a nice way with any operator.\n",
      "\n",
      "A complete list of these methods, with descriptions, is available at http://docs.python.org/2/reference/datamodel.html#special-method-names\n",
      "\n",
      "A few examples are\n",
      "\n",
      "    __add__\n",
      "    __eq__\n",
      "    __and__\n",
      "    __str__"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "class A(object):\n",
      "    a = []\n",
      "\n",
      "\n",
      "class B(A):\n",
      "    def __init__(self, name):\n",
      "        self.name = name\n",
      "        self.a.append(self.name)\n",
      "    \n",
      "    def print_list(self):\n",
      "        print(self.a)\n",
      "        \n",
      "    def __del__(self):\n",
      "        self.a.remove(self.name)\n",
      "        \n",
      "    def __add__(self, other):\n",
      "        combined_name = \"\".join(sorted([self.name, other.name]))\n",
      "        combined_b = B(combined_name)\n",
      "        return combined_b\n",
      "\n",
      "b1 = B(\"1\")\n",
      "b2 = B(\"2\")"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 38
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b12 = b2 + b1"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 39
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b1.print_list()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "['1', '2', '12']\n"
       ]
      }
     ],
     "prompt_number": 37
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "It is usually a good design choice to make the `repr` of an object behave as the input of the constructur to create an equivalent object."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "# NumPy\n",
      "\n",
      "NumPy is the heart of most calculations when using Python for scientific computing.\n",
      "\n",
      "The most important part of it is that it provides a very efficient multidimensional array object"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 40
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "The difference between an `array` and a `list` is that an `array` puts data in to contiguous memory blocks. While a `list` has a block of addresses referring to data.\n",
      "\n",
      "This means `lists` are dynamic, you can store anything of any size in them, and grow them and shrink them however you want.\n",
      "\n",
      "NumPy `array`s can only have one datatype per instance. They _can_ be grown and shrunk in size, but that is a (relatively) costly operation and should be avoided.\n",
      "\n",
      "To instantiate an array one can pass a list to its constructor. But there are also efficient methods to create commonly needed arrays."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "a = np.array([1., 2., 3.])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 41
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "a"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 42,
       "text": [
        "array([ 1.,  2.,  3.])"
       ]
      }
     ],
     "prompt_number": 42
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Using the efficient array data structure, with numpy one can do _broadcasted_ operations on the arrays"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "a * 4.0"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 44,
       "text": [
        "array([  4.,   8.,  12.])"
       ]
      }
     ],
     "prompt_number": 44
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "type(a)"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 45,
       "text": [
        "numpy.ndarray"
       ]
      }
     ],
     "prompt_number": 45
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Arrays can have any number of dimensions, unlike a list which onle has one. (n-dimensional array)\n",
      "\n",
      "Instantiated by lists of lists"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b = np.array([[1, 4, 6], [5, 2, 2]])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 46
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 47,
       "text": [
        "array([[1, 4, 6],\n",
        "       [5, 2, 2]])"
       ]
      }
     ],
     "prompt_number": 47
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "We can access items like on a list of lists"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b[1][0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 49,
       "text": [
        "5"
       ]
      }
     ],
     "prompt_number": 49
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "But arrays also have their own multidimensional accessors "
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b[1, 0]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 50,
       "text": [
        "5"
       ]
      }
     ],
     "prompt_number": 50
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "As well as multidimensional slicing"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b[:, 1:2]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 51,
       "text": [
        "array([[4],\n",
        "       [2]])"
       ]
      }
     ],
     "prompt_number": 51
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Beware though that the ndarray slices are _views_ in to the array rather than copies!\n",
      "\n",
      "This means one can also broadcast on to what the view is referring to"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b[1:2, 1:2] = 0"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 58
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 59,
       "text": [
        "array([[1, 0, 6],\n",
        "       [5, 0, 2]])"
       ]
      }
     ],
     "prompt_number": 59
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "b.shape"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "pyout",
       "prompt_number": 60,
       "text": [
        "(2, 3)"
       ]
      }
     ],
     "prompt_number": 60
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": 60
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Good overview of most things one would need to know about arrays to use them efficiently:\n",
      "\n",
      "- http://www.scipy.org/Tentative_NumPy_Tutorial\n",
      "\n",
      "When operations are performed in a _vectorized_ way, specialized C and Fortran methods will be used to perform the operation. Making them very quick and efficient."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Assignment\n",
      "\n",
      "- Make a new file in your module called `session2.py`, and move the code from `__init__.py` in to it.\n",
      "\n",
      "- Change your `getting_data.py` to import in this fashion: `from lastname.session2 import ...`\n",
      "\n",
      "- Make a context manager which changes the current directory for the python script using it, and changes back to where you came from upon exit.\n",
      "\n",
      "(Hint, look up `os.chdir` here http://docs.python.org/2/library/os.html )\n",
      "\n",
      "Implement a class `CourseRepo` which takes a \"surname\" string in the constructor.\n",
      "\n",
      "The class should have an attribute \"required\" which is a list of these strings:\n",
      "\n",
      "- `\".git\"`\n",
      "- `\"setup.py\"`\n",
      "- `\"README.md\"`\n",
      "- `\"scripts/getting_data.py\"`\n",
      "- `\"scripts/check_repo.py\"`\n",
      "- `\"lastname/__init__.py\"`\n",
      "- `\"lastname/session3.py\"`\n",
      "\n",
      "Where `\"lastname\"` is the \"surname\" string you gave to the constructor\n",
      "\n",
      "`surname` should be a **property**, such that when it is set, the `required` attribute changes to reflect the new surname.\n",
      "\n",
      "Example:\n",
      "\n",
      "    repo = CourseRepo(\"a\")\n",
      "    print(repo.required[-1])\n",
      "    # prints a/session3.py\n",
      "    \n",
      "    repo.surname = \"b\"\n",
      "    print(repo.required[-1])\n",
      "    # prints b/session3.py\n",
      "\n",
      "The class should have a method `check()` which shall return `True` if all the strings in that list are existing files or directories.\n",
      "\n",
      "(Hint: `os.path.exists`)\n",
      "\n",
      "The context manager and class should be implemented in a file `session3.py` in your `lastname` module.\n",
      "\n",
      "Then make a script, `scripts/check_repo.py`, which imports the context manager for changing current directory as well as the `CourseRepo` class from the module. Like this:\n",
      "\n",
      "    from lastname.session3 import CourseRepo, repo_dir\n",
      "\n",
      "(where `repo_dir` is the name of the context manager)\n",
      "\n",
      "This script should take an argument which is the absolute path to a repository.\n",
      "\n",
      "(Hint: `sys.argv` or the built in `argparse` module)\n",
      "\n",
      "The script should change directory to this given directory using the context manager. It should make an instance of `CourseRepo` using the final part of the absolute path, and call the `check()` method. If `check()` returns `True` the script shall print \"PASS\", otherwise the script should print \"FAIL\".\n",
      "\n",
      "Example: If I call the script like\n",
      "\n",
      "    $ check_repo.py /Home/user/a\n",
      "\n",
      "it should make a `CourseRepo` instance like in the example above (with `\"a\"`)\n",
      "\n",
      "Like the other script, this script should also be installed when running `python setup.py install`.\n",
      "\n",
      "Note: If your repo has a structure this script is not expecting, fixing it should be rather smooth using `git mv` for renaming/moving files and directories."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}