ODSC.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Teaching Data Science Effectively\n",
    "\n",
    "### Robert Schroll\n",
    "### The Data Incubator\n",
    "\n",
    "Follow along at [bit.ly/odsc-tdi](https://bit.ly/odsc-tdi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data Scientists Needed\n",
    "\n",
    "Quanthub estimates a shortage of 250,000 data scientists in 2020.\n",
    "\n",
    "Few are coming directly from academia."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "notes"
    }
   },
   "source": [
    "[Quanthub's survey](https://quanthub.com/data-scientist-shortage-2020/) is based on data from 2019 and 2020, so it probably does not fully reflect the effects of the pandemic.  But hiring remains strong, and this estimate is consistent with the order of magnitude from other research."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data Science Skills Needed\n",
    "\n",
    "Many who don't consider themselves data scientists benefit from data science skills.\n",
    "\n",
    "Pandas can be a game-changer, by automating spreadsheet tasks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Data Science Skills\n",
    "\n",
    "Data Scientists need to combine skills in\n",
    "1. Data analysis\n",
    "2. Programming\n",
    "3. Communication"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## About the Data Incubator\n",
    "\n",
    "The Data Incubator trains recent graduates and experienced workers.\n",
    "\n",
    "Our Data Science Fellowship is an 8-week bootcamp.\n",
    "- 1000+ students trained in the basics of data science.\n",
    "- Students placed at 100+ companies, including Genentech, Square Space, and Freddie Mac\n",
    "\n",
    "We run private, customized trainings for dozens of companies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## About Me\n",
    "\n",
    "I have been programming and conducting data analysis for most of this century.\n",
    "\n",
    "First as a physicist, now as a data scientist.\n",
    "\n",
    "I have been teaching data science for 5 years."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Teaching Data Science Effectively\n",
    "\n",
    "1. Jupyter as a pedagogical tool\n",
    "2. Learning by doing\n",
    "3. The importance of failure\n",
    "4. Where we're still learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Jupyter as a Pedagogical Tool\n",
    "\n",
    "Jupyter notebooks are a great tool for data science.\n",
    "\n",
    "They also work well for teaching."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Mixing Languages\n",
    "\n",
    "Data scientists are often moving between multiple languages and tools.\n",
    "\n",
    "Jupyter and the IPython kernel allow you to mix many of these tools in a single interface."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(10):\n",
    "    print('Hi! ' * i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Bash\n",
    "\n",
    "Shell code can be run with the `!` shortcut."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! grep \"Bash\" ODSC.ipynb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Bash\n",
    "\n",
    "There is also a bash magic, for longer code sections."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "\n",
    "for f in *; do\n",
    "    echo \"Hi $f\"\n",
    "done"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## SQL\n",
    "\n",
    "The [IPython SQL magic](https://github.com/catherinedevlin/ipython-sql) allows SQL code directly in cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext sql\n",
    "%sql sqlite:///small_data/customers.sqlite"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%sql\n",
    "\n",
    "SELECT * FROM customers LIMIT 5;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## HTML/JS\n",
    "\n",
    "Jupyter already has tools to display HTML documents, but our [iHTML](https://github.com/thedataincubator/ihtml) package lets you demo HTML code directly in a notebook.\n",
    "\n",
    "First, we'll create a Javascript document:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ihtml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%jsdoc clicker\n",
    "document.addEventListener(\"DOMContentLoaded\", function(e) {\n",
    "    document.querySelector(\"h1\").addEventListener(\"click\", function(event) {\n",
    "        var div = document.createElement(\"div\");\n",
    "        div.textContent = \"Hi!\";\n",
    "        document.body.appendChild(div);\n",
    "    })\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## HTML/JS\n",
    "\n",
    "Then we can use this inside a HTML document:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "%%ihtml 200\n",
    "<html>\n",
    "    <head>\n",
    "        <style>\n",
    "            body { background: #eee; }\n",
    "        </style>\n",
    "        {{ clicker | jsdoc }}\n",
    "    </head>\n",
    "    <body>\n",
    "        <h1>Click me!</h1>\n",
    "    </body>\n",
    "</html>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Benefits of Jupyter Notebooks\n",
    "\n",
    "**Interactivity:** Cause and effect are tightly coupled.\n",
    "\n",
    "**Modifiability:** Easy to make small changes and see effects.\n",
    "\n",
    "Together, these encourage experimentation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Deploying Jupyter for Everyone\n",
    "\n",
    "[Jupyter](https://jupyter.org/) provides a server for a single user.\n",
    "\n",
    "[JupyterHub](https://jupyter.org/hub) provides many users their own servers.\n",
    "\n",
    "[Zero-to-JupyterHub](https://zero-to-jupyterhub.readthedocs.io/en/latest/) runs JupyterHub on a K8s cluster.\n",
    "\n",
    "All for free, with a wonderful community!\n",
    "\n",
    "For details on our set up, see [our DigitalOcean Tech Talk](https://www.digitalocean.com/community/tech_talks/scaling-a-school-bringing-data-science-curriculum-to-20-000-students-in-the-cloud)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Learning by Doing\n",
    "\n",
    "Promote action at four levels:\n",
    "1. Prepared interactive elements\n",
    "2. Small inline coding exercises\n",
    "3. Stand-alone \"miniprojects\"\n",
    "4. Capstone project"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Prepared Interactive Elements\n",
    "\n",
    "### excerpt from _Bias, Variance, and Overfitting_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "In this notebook, we will illustrate the notion of overfitting and learn how we can try to prevent it.  We'll use data describing daily sales in a store given the number of customers in the store that day. Let's start by plotting the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": null
   },
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "sns.set()\n",
    "\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "df_sales = pd.read_csv('small_data/SalesData.csv')\n",
    "\n",
    "X = df_sales[['Customers']]\n",
    "y = df_sales['Sales']\n",
    "\n",
    "plt.plot(X, y, 'k.')\n",
    "plt.title('Daily Store Sales Given Number of Customers That Day')\n",
    "plt.xlabel('Number of customers')\n",
    "plt.ylabel('Store sales [dollars]');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "## Train a decision tree model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "We want to predict store sales given the number of customers in the store. We will use a decision tree model to illustrate an example of the bias-variance tradeoff. Decision trees are used to solve both regression and classification problems. A decision tree model makes a prediction by using a flowchart-like binary tree structure that is constructed during training. \n",
    "\n",
    "One of the hyperparameters of a decision tree is `max_depth` that controls the maximum depth of the tree from its root.  Let's experiment with trees of different depths by using a widget that will allow us to pick different `max_depth` values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": null
   },
   "outputs": [],
   "source": [
    "from ipywidgets import interactive, IntSlider\n",
    "from sklearn.tree import DecisionTreeRegressor\n",
    "\n",
    "def train_and_plot(max_depth):\n",
    "    est = DecisionTreeRegressor(max_depth=max_depth)\n",
    "    est.fit(X, y)\n",
    "\n",
    "    plt.figure(figsize=(9,6))\n",
    "    plt.plot(X, y, 'k.', label='data')\n",
    "    line = plt.plot(X, est.predict(X), c='#ba2121ff', label='model')\n",
    "    plt.setp(line, linewidth=3., alpha=0.7)\n",
    "    plt.title('Daily Store Sales Given Number of Customers That Day')\n",
    "    plt.xlabel('Number of customers')\n",
    "    plt.ylabel('Store sales [dollars]');\n",
    "    plt.legend(loc='upper left')\n",
    "    plt.show()\n",
    "    \n",
    "max_depth_slider = IntSlider(min=1, max=10, step=1, value=2)\n",
    "interactive(train_and_plot, max_depth=max_depth_slider)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "### Exercise\n",
    "\n",
    "Play around with `max_depth` slider in the above plot.  What happens when you set it to `max_depth=5`.  Does the fitted model look better or worse?  What about for `max_depth=2` or `max_depth=3`? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Inline Coding Exercises\n",
    "\n",
    "### excerpt from _Basic Neural Networks_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": null
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "### The XOR problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "Research into artificial neurons dates to the late 40s, but it was not until 1969 that Marvin Minsky and Seymour Papert pointed out that basic neurons were unable to reproduce the **exclusive-or** (XOR) function.  This Boolean function of two Boolean variables returns true if exactly one of its inputs is true:\n",
    "\n",
    "$$ \\mathrm{XOR}(0, 0) = \\mathrm{XOR}(1, 1) = 0 \\ \\ \\ \\ \\ \\ \\mathrm{XOR}(0, 1) = \\mathrm{XOR}(1, 0) = 1 $$\n",
    "\n",
    "Below, we create a related two-class classification problem, with one class clustered about (0, 0) and (1, 1), and the other about (0, 1) and (1, 0).  It would be quite easy to draw a boundary separating the two classes by hand. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": null
   },
   "outputs": [],
   "source": [
    "centers = np.array([[0, 0]] * 100 + [[1, 1]] * 100\n",
    "                   + [[0, 1]] * 100 + [[1, 0]] * 100)\n",
    "np.random.seed(42)\n",
    "data = (np.random.normal(0, 0.2, (400, 2)) + centers).astype(np.float32)\n",
    "labels = np.array([[0]] * 200 + [[1]] * 200).astype(np.float32)\n",
    "\n",
    "plt.scatter(data[:,0], data[:,1], c=labels.ravel(), cmap=plt.cm.RdYlBu)\n",
    "plt.colorbar();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "The math behind this isn't as bad as it might seem at first.  All of the weights of the neurons in the hidden layer can be combined into a single $2\\times2$ matrix $W^{(1)}$.  The final neuron's weights will be in a $2\\times1$ matrix $W^{(2)}$.  The biases behave similarly.  Then our final probabilistic prediction is just\n",
    "\n",
    "$$ p_j = f_2\\bigg( f_1\\left( X_{ji} W^{(1)}_{ik} + b^{(1)}_k \\right) W^{(2)}_k + b^{(2)} \\bigg).$$\n",
    "\n",
    "We are using the Einstein notation: All repeated indices are implicitly summed over.  Both $f_1$ and $f_2$ represent the logistic function, which is taken to operate element-wise over tensors.\n",
    "\n",
    "The **backpropagation** algorithm, developed by Paul Werbos in 1975, points out that we can use gradient descent (or similar algorithms) to optimize all of the parameters in these sorts of expressions.  All it takes is successive applications of the chain rule.  In fact, there's nothing special we have to do to make use of it: TensorFlow's optimizers automatically work though the successive derivatives to generate the update rules.  All we have to do is set up the calculation:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will build a simple neural network with a single hidden layer.  Tensorflow's automatic differentiation will handle the backpropagation step for us."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": null
   },
   "outputs": [],
   "source": [
    "tf.random.set_seed(42)  #  set seed for reproducibility\n",
    "\n",
    "hidden_size = 2\n",
    "\n",
    "layer_shapes = [(2, hidden_size), (hidden_size, 1)]\n",
    "\n",
    "#  set up W and b of the appropriate sizes for each layer\n",
    "layers = [(tf.Variable(tf.random.normal(shape)), tf.Variable(tf.zeros(shape[1])))\n",
    "         for shape in layer_shapes]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "We also need to redefine the `logits` function to account for the hidden layer (and therefore redefine `loss` for the new definition of `logits`).  The `predict` and `score` functions are the same as for logistic regression. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": null
   },
   "outputs": [],
   "source": [
    "def logits(X):\n",
    "    _ = X\n",
    "    for W, b in layers[:-1]:\n",
    "        _ = tf.nn.sigmoid(tf.matmul(_, W) + b)\n",
    "        \n",
    "    W, b = layers[-1]\n",
    "    return tf.matmul(_, W) + b\n",
    "\n",
    "def loss(X, y):\n",
    "    def loss_():\n",
    "        return tf.reduce_mean(\n",
    "            tf.nn.sigmoid_cross_entropy_with_logits(logits=logits(X), labels=y))\n",
    "    \n",
    "    return loss_\n",
    "\n",
    "def predict_proba(X):\n",
    "    return tf.nn.sigmoid(logits(X))\n",
    "\n",
    "def predict(X):\n",
    "    return tf.cast(predict_proba(X) > 0.5, tf.float32)\n",
    "    \n",
    "def score(X, y):\n",
    "    return tf.reduce_mean(tf.cast(tf.equal(predict(X), y), tf.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "And let's run it.  We need more steps to get all of the weights well-trained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": null
   },
   "outputs": [],
   "source": [
    "steps = 300\n",
    "rounds = 10\n",
    "\n",
    "eta = 0.5\n",
    "\n",
    "opt = tf.keras.optimizers.SGD(learning_rate=eta)\n",
    "\n",
    "for i in range(1, rounds+1):\n",
    "    for j in range(steps):\n",
    "        opt.minimize(loss(data, labels), [var for layer in layers for var in layer])\n",
    "    print(f'Round {i}:')\n",
    "    print(f'   Loss: {loss(data, labels)()}')\n",
    "    print(f'   Accuracy: {score(data, labels)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "We can verify the improved accuracy by examining our predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "slideshow": null
   },
   "outputs": [],
   "source": [
    "mesh = np.column_stack([a.reshape(-1) for a in np.meshgrid(np.r_[-1:2:100j], np.r_[-1:2:100j])]).astype(np.float32)\n",
    "ymesh = predict_proba(mesh).numpy()\n",
    "\n",
    "plt.imshow(ymesh.reshape(100,100), cmap=plt.cm.RdYlBu, origin='lower',\n",
    "           extent=(-1, 2, -1, 2), vmin=0, vmax=1)\n",
    "plt.scatter(data[:, 0], data[:, 1], c=labels.ravel(), cmap=plt.cm.RdYlBu,\n",
    "            edgecolor='w', lw=1)\n",
    "plt.axis((-1, 2, -1, 2))\n",
    "plt.colorbar();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "### Exercise: Number of hidden neurons"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": null
   },
   "source": [
    "Change the number of neurons in the hidden layer.  How does this change the predictions made by the model?  What happens when you add many neurons to this hidden layer?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Miniprojects\n",
    "\n",
    "### excerpt from _Assignment 1_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import grader"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Other questions will require you to build a function that takes some input and returns a specified output. Build a function that takes a list of numbers as input and returns a list of their squares."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def square(x):\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the `grader.score` method will ask the grader for the input, pass it to your function, and report the output back for grading. Before we submit our answer to the grader, let's discuss `grader.check`.\n",
    "\n",
    "For some questions, we have checks along the way. These checks let you know if there's any potential problems with your solution. If the expression in `grader.check` is `True`, it will return `True`. If not, it will raise a `ValueError` and we should see how we can resolve the issue. In the check below, we make sure that the length of the output list is the same as the input list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check to see that the input and output list are the same length.\n",
    "grader.check(len(square([0, 1, 2])) == 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have passed the check, let's submit our `square` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grader.score(\"assignment1__square\", square)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Miniproject: New York Social Graph\n",
    "\n",
    "New York Social Diary hosted 1000+ pages of photos.\n",
    "\n",
    "[Each page](https://web.archive.org/web/20150913112351/http://www.newyorksocialdiary.com/party-pictures/2014/holiday-dinners-and-doers) has photographs with names in captions.\n",
    "\n",
    "We can understand the social network by:\n",
    "- Finding all photo pages\n",
    "- Scraping all captions\n",
    "- Detecting names in captions\n",
    "- Building a connectivity graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Miniproject Feedback\n",
    "\n",
    "Automatic grader provides feedback on answer, not code style.\n",
    "\n",
    "Instructor-led code review sessions review student-submitted code.\n",
    "\n",
    "Deep focus on one part of the project.\n",
    "- Compare different approaches.\n",
    "- Fix bad coding practices."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Capstone Project\n",
    "\n",
    "Data scientists must solve the *right* problems and communicate the results.\n",
    "\n",
    "Students design and execute a full data science project.\n",
    "\n",
    "Usual problems are not technical, but around use cases.\n",
    "\n",
    "Some recent projects:\n",
    "- Finding broadband opportunities for under-served communities in Tennessee.\n",
    "- Identifying useful product reviews.\n",
    "- Finding shelter pets for adoption by image search."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## The Importance of Failure\n",
    "\n",
    "You need both successes and failures to train a ML model.\n",
    "\n",
    "Students learn as much from failures as from successes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## The Benefits of Failure\n",
    "\n",
    "Failure is the default state of programming.  Most time is spent debugging.\n",
    "\n",
    "The marginal cost of code is $0. Known solutions would already be in a library.\n",
    "\n",
    "Students need to learn how to find their way out of problems.\n",
    "\n",
    "Hand-holding does not promote self-sufficiency."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Google-fu is a Skill\n",
    "\n",
    "<img src=\"https://i.redd.it/7lfrc6p5xna21.png\" width=\"70%\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Designing for Failure\n",
    "\n",
    "More than just fill-in-the-blank.\n",
    "\n",
    "Problems should involve aspects not covered previously.\n",
    "\n",
    "Projects should have multiple roots to success."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Helping Students Overcome Failure\n",
    "\n",
    "Provide sketches or outlines of potential solutions.\n",
    "\n",
    "Checkpoints let students check their own work.\n",
    "\n",
    "Make instructors available:\n",
    "- Lecture\n",
    "- Office hours\n",
    "- One-on-one meetings\n",
    "- Slack"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Instructor Failures are Teaching Moments\n",
    "\n",
    "Students learn the most when the instructor gets stuck.\n",
    "\n",
    "Yes, some _schadenfreude_.\n",
    "\n",
    "But also, an invaluable chance to teach debugging skills.\n",
    "\n",
    "$\\Rightarrow$ Get into trouble while teaching!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Still Learning: Demoing Failure\n",
    "\n",
    "Prepared failures don't resonate as well.\n",
    "\n",
    "It's hard to fake the panic of facing a bewildering bug.\n",
    "\n",
    "Part of bug hunting is going down, and then abandoning blind alleys."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Still Learning: Teaching Generalization\n",
    "\n",
    "Some students want a flow chart to follow.\n",
    "\n",
    "Repeat examples step by step, without understanding logic.\n",
    "\n",
    "These students struggle mightily on miniprojects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Still Learning: Magical Thinking\n",
    "\n",
    "Some students treat code as a magical incantation.\n",
    "\n",
    "Rearrange syntax until it runs.\n",
    "\n",
    "They seem uninterested in understanding *why* code failed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Find Out More\n",
    "\n",
    "- Data Science Fellowship\n",
    "- Data Engineering Fellowship\n",
    "- Private Training\n",
    "\n",
    "[thedataincubator.com](https://www.thedataincubator.com)\n",
    "\n",
    "Robert Schroll &mdash; robert@thedataincubator.com\n",
    "\n",
    "Visit us at booth #9!\n",
    "\n",
    "Slides: [bit.ly/odsc-tdi](https://bit.ly/odsc-tdi)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "rise": {
   "overlay": "<div class='tdi-bg'></div><div class='tdi-footer'><span class='left'>pragmaticinstitute.com</span><span class='right'>Pragmatic Institute data science powered by The Data Incubator</span></div>",
   "scroll": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}