02-1. 用CNN圖形辨識（還是MNIST）.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 主題 02-1. 用 CNN 做圖形辨識\n",
    "\n",
    "Yann LeCun 被譽為 Deep Learning 的三巨頭之一。他的 CNN (Convolutional Neural Networks) 是讓 Neural Network 重新受到重視的主因之一。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 初始準備\n",
    "\n",
    "基本上和之前是一樣的, 我們就不再說明。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "env: KERAS_BACKEND=tensorflow\n"
     ]
    }
   ],
   "source": [
    "%env KERAS_BACKEND=tensorflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 讀入 MNIST 數據庫\n",
    "\n",
    "### 2.1 由 Keras 讀入 MNIST\n",
    "\n",
    "基本上和我們上次一樣, 這次因為 Keras 已偷偷把數據庫存在你的電腦, 所以會快很多!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
<<<<<<< HEAD
      "/Users/yenlung/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n",
=======
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 輸入格式整理\n",
    "\n",
    "如果你還記得, 我們每筆輸入資料都是 28x28 的陣列, CNN 其實就是吃「圖」的, 所以基本上不用像之前把每筆資料拉平。「但。是。」平常的圖都有 R, G, B 三個 channels, 每個 channel 都是一個矩陣, 也就是一張圖可能是三個矩陣! 我們是灰階, 也就是只有一個 channel。但這件事也要明確的告訴 Keras。\n",
    "\n",
    "換句話說, 我們的輸入每筆資料型式要從 (28, 28) 換成 (28, 28, 1)!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train.reshape(60000, 28, 28, 1)\n",
    "x_test = x_test.reshape(10000, 28, 28, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "確認一下..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(28, 28, 1)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train[9487].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原來 28x28 矩陣..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 240,\n",
       "        253, 165,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  32, 241,\n",
       "        252, 252,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   2, 141, 252,\n",
       "        252, 252,  98,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  30, 252, 252,\n",
       "        252, 252, 149,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  21, 217, 252,\n",
       "        252, 252, 149,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 134, 252,\n",
       "        252, 252, 223,  31,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 134, 252,\n",
       "        252, 252, 253,  44,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   8, 239,\n",
       "        252, 252, 253,  44,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 238,\n",
       "        252, 252, 253,  44,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 238,\n",
       "        252, 252, 253,  44,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 178,\n",
       "        253, 253, 255, 106,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  90,\n",
       "        252, 252, 253, 206,  21,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  90,\n",
       "        252, 252, 253, 252,  88,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  90,\n",
       "        252, 252, 253, 252,  88,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  63,\n",
       "        234, 252, 253, 252, 220,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "        176, 252, 253, 252, 237,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "         45, 252, 253, 252, 245,  71,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "         29, 215, 253, 252, 237,   8,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0, 149, 253, 252, 237,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,  61, 165, 252, 237,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0]], dtype=uint8)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train[9487][:,:,0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=x_train[9400][:,:,0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(28, 28)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7ff963e73e48>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADqFJREFUeJzt3X2MVPW9x/HPVx5iEHwKy4MW3FKJuUQTuJmQm3h9uDZWq02g0SoEG0hqt4n1gaTGC8SkPkAixpZLIqmhdgMmlLZQvSCBa4250Yu5aRyNqRbuvcW6FATZJVQLEiQL3/vHHpoVd36zO3Nmzizf9ysxO3O+58z5etjPnpn5zZmfubsAxHNe0Q0AKAbhB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8Q1Mhm7mz8+PHe3t7ezF0CoXR1denw4cM2mHXrCr+Z3SpptaQRkp5396dS67e3t6tcLtezSwAJpVJp0OvW/LTfzEZIWiPpm5JmSJpvZjNqfTwAzVXPa/7Zkva4+5/d/aSkX0mak09bABqtnvBfLmlfv/v7s2VfYGYdZlY2s3JPT08duwOQp3rCP9CbCl+6Ptjd17p7yd1LbW1tdewOQJ7qCf9+SVP63f+KpAP1tQOgWeoJ/1uSppvZV81stKR5krbm0xaARqt5qM/de83sfkmvqG+or9Pd/5hbZwAaqq5xfnffLml7Tr0AaCI+3gsERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQdc3Sa2Zdko5KOiWp191LeTR1runt7U3Wd+3alaxv3bq15n13dnYm611dXcn6E088kazfcccdyfoVV1xRsTZmzJjktmisusKf+Rd3P5zD4wBoIp72A0HVG36X9Dsze9vMOvJoCEBz1Pu0/1p3P2BmEyS9amb/4+5v9F8h+6PQIUlTp06tc3cA8lLXmd/dD2Q/uyW9JGn2AOusdfeSu5fa2trq2R2AHNUcfjO7wMzGnbkt6RuS3s+rMQCNVc/T/omSXjKzM4/zS3f/j1y6AtBw5u5N21mpVPJyudy0/TXL888/n6y//PLLyfq2bdvybCdX1X4/sj/+FS1YsKBibd26dcltzzuPwaihKpVKKpfL6X+UDEcXCIrwA0ERfiAowg8ERfiBoAg/EFQeV/WdE44ePZqsL1q0qGJty5YtyW3rHU598MEHk/Xzzz+/rsdPWblyZV3bb9iwoWJt3rx5yW1vu+22uvaNNM78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxBUmHH+EydOJOu33357sv7mm29WrF155ZXJbdesWZOs33DDDcn6yJHpf6Zql9XWY/Hixcn6ZZddVvNjf/zxxzVvi/px5geCIvxAUIQfCIrwA0ERfiAowg8ERfiBoMKM81e75n358uXJ+o4dOyrWVqxYkdx2OH8F9Weffdawx2b6tmIN399KAHUh/EBQhB8IivADQRF+ICjCDwRF+IGgqo7zm1mnpG9J6nb3q7Nll0r6taR2SV2S7nL3vzauzca7/vrr66qfq+r9/07Nd3DTTTfV9dioz2DO/Osk3XrWsiWSXnP36ZJey+4DGEaqht/d35B05KzFcyStz26vlzQ3574ANFitr/knuvtBScp+TsivJQDN0PA3/Mysw8zKZlbu6elp9O4ADFKt4T9kZpMlKfvZXWlFd1/r7iV3L7W1tdW4OwB5qzX8WyUtzG4vlJSephZAy6kafjPbKOm/JV1lZvvN7HuSnpJ0s5n9SdLN2X0Aw0jVcX53n1+h9PWce0EDVPtu/McffzxZP3DgQLI+bdq0ZP2ZZ56pWBvO33NwLuDoA0ERfiAowg8ERfiBoAg/EBThB4IK89XdrezIkbOvm/qizz//PFl//fXXK9aWLElfcLlv375kvZoPP/wwWd+4cWPF2p133pncdsIELhlpJM78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxCUuXvTdlYqlbxcLjdtf63i0KFDyfqMGTOS9U8++STPdobk9OnTyXo9l+WOHj06We/o6EjWH3744WR9ypQpQ+5puCuVSiqXyzaYdTnzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQXM/fBMePH0/WGzmOf++99ybrU6dOTdYvvvjiZH3ixInJ+ubNmyvWNm3alNz22WefTdbXr1+frK9Zs6ZibcGCBcltI+DMDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBVb2e38w6JX1LUre7X50te0zS9yX1ZKstc/ft1XYW9Xr+Y8eOJes7duyo6/Gvu+66irVJkybV9diN9MEHHyTrW7ZsSdYfffTRZP3EiRMVa/fdd19y21WrViXro0aNStaLkvf1/Osk3TrA8lXuPjP7r2rwAbSWquF39zckpaeUATDs1POa/34z+4OZdZrZJbl1BKApag3/zyR9TdJMSQcl/aTSimbWYWZlMyv39PRUWg1Ak9UUfnc/5O6n3P20pJ9Lmp1Yd627l9y91NbWVmufAHJWU/jNbHK/u9+W9H4+7QBolqqX9JrZRkk3ShpvZvsl/VjSjWY2U5JL6pL0gwb2CKAB+N5+DFuvvPJKsj537tyKtZMnTya3Xb58ebK+dOnSZL0ofG8/gKoIPxAU4QeCIvxAUIQfCIrwA0Ex1Idz1sqVKyvWli1bltz2oosuStb37t2brI8bNy5ZbxSG+gBURfiBoAg/EBThB4Ii/EBQhB8IivADQTFFN85Zt9xyS8VatXH+Tz/9NFk/depUTT21Es78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4/wYto4fP56sv/jii03qZHjizA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQVUd5zezKZJekDRJ0mlJa919tZldKunXktoldUm6y93/2rhWca7p7e1N1jdv3pysr1ixIlnftWvXkHs645FHHknWx44dW/Njt4rBnPl7Jf3I3f9B0j9J+qGZzZC0RNJr7j5d0mvZfQDDRNXwu/tBd38nu31U0m5Jl0uaI2l9ttp6SXMb1SSA/A3pNb+ZtUuaJen3kia6+0Gp7w+EpAl5NwegcQYdfjMbK+m3kha7+9+GsF2HmZXNrNzT01NLjwAaYFDhN7NR6gv+Bnc/c7XEITObnNUnS+oeaFt3X+vuJXcvtbW15dEzgBxUDb+ZmaRfSNrt7j/tV9oqaWF2e6GkLfm3B6BRBnNJ77WSvivpPTN7N1u2TNJTkn5jZt+T9BdJ32lMi63vxIkTyfrdd9+drK9evTpZb29vH2pLuak2hfvOnTuT9U2bNtVUk6Tu7gGfTA7a6NGjK9Yeeuih5LbLly9P1keMGFFTT62kavjdfaekSvN9fz3fdgA0C5/wA4Ii/EBQhB8IivADQRF+ICjCDwTFV3fnoKurK1nftm1bsr5nz55kffv27cl6ajz7o48+Sm5b7bLZo0ePJuvPPfdcsl6PMWPGJOv33HNPsr506dKKtalTp9bU07mEMz8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBMU4fw6mT5+erFf7Guinn346WZ82bdqQe8pLtbH2ap9BOHLkSMXarFmz6to3Y/X14cwPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Exzp+Dat/h/uSTTybrV111VbK+d+/eZH337t0Va9dcc01y22oeeOCBZP3CCy+s6/FRHM78QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxBU1XF+M5si6QVJkySdlrTW3Veb2WOSvi+pJ1t1mbunL+4OauTI9GFetGhRcxoB+hnMh3x6Jf3I3d8xs3GS3jazV7PaKnd/pnHtAWiUquF394OSDma3j5rZbkmXN7oxAI01pNf8ZtYuaZak32eL7jezP5hZp5ldUmGbDjMrm1m5p6dnoFUAFGDQ4TezsZJ+K2mxu/9N0s8kfU3STPU9M/jJQNu5+1p3L7l7qa2tLYeWAeRhUOE3s1HqC/4Gd39Rktz9kLufcvfTkn4uaXbj2gSQt6rhNzOT9AtJu939p/2WT+632rclvZ9/ewAaZTDv9l8r6buS3jOzd7NlyyTNN7OZklxSl6QfNKRDAA0xmHf7d0qyAUqM6QPDGJ/wA4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBGXu3rydmfVI6j/f9HhJh5vWwNC0am+t2pdEb7XKs7cr3H1Q35fX1PB/aedmZXcvFdZAQqv21qp9SfRWq6J642k/EBThB4IqOvxrC95/Sqv21qp9SfRWq0J6K/Q1P4DiFH3mB1CQQsJvZrea2f+a2R4zW1JED5WYWZeZvWdm75pZueBeOs2s28ze77fsUjN71cz+lP0ccJq0gnp7zMw+yo7du2Z2W0G9TTGz/zSz3Wb2RzN7KFte6LFL9FXIcWv6034zGyHp/yTdLGm/pLckzXf3XU1tpAIz65JUcvfCx4TN7HpJxyS94O5XZ8uelnTE3Z/K/nBe4u7/2iK9PSbpWNEzN2cTykzuP7O0pLmSFqnAY5fo6y4VcNyKOPPPlrTH3f/s7icl/UrSnAL6aHnu/oakI2ctniNpfXZ7vfp+eZquQm8twd0Puvs72e2jks7MLF3osUv0VYgiwn+5pH397u9Xa0357ZJ+Z2Zvm1lH0c0MYGI2bfqZ6dMnFNzP2arO3NxMZ80s3TLHrpYZr/NWRPgHmv2nlYYcrnX3f5T0TUk/zJ7eYnAGNXNzswwws3RLqHXG67wVEf79kqb0u/8VSQcK6GNA7n4g+9kt6SW13uzDh85Mkpr97C64n79rpZmbB5pZWi1w7Fppxusiwv+WpOlm9lUzGy1pnqStBfTxJWZ2QfZGjMzsAknfUOvNPrxV0sLs9kJJWwrs5QtaZebmSjNLq+Bj12ozXhfyIZ9sKOPfJI2Q1OnuK5rexADMbJr6zvZS3ySmvyyyNzPbKOlG9V31dUjSjyX9u6TfSJoq6S+SvuPuTX/jrUJvN6rvqevfZ24+8xq7yb39s6T/kvSepNPZ4mXqe31d2LFL9DVfBRw3PuEHBMUn/ICgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBPX/f104vfdrAggAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(X, cmap='Greys')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train[9400]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 輸出格式整理\n",
    "\n",
    "和上次一樣, 我們用標準 1-hot 方式處理。"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 8,
=======
   "execution_count": 12,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.utils import np_utils"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 9,
=======
   "execution_count": 13,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = np_utils.to_categorical(y_train, 10)\n",
    "y_test = np_utils.to_categorical(y_test, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 0., 1., 0., 0., 0., 0., 0., 0.], dtype=float32)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train[9400]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 打造你的 CNN\n",
    "\n",
    "### 3.1 決定神經網路架構、讀入相關套件\n",
    "\n",
    "CNN 我們一樣要決定用幾層的 CNN, 然後是不是每次都要做 max-pooling。再來就是拉平、送入標準神經網路 (再度要決定幾層、幾個神經元)。\n",
    "\n",
    "* 做 <span style=\"color:red;\">3</span> 次 convolution, 每次都接 max-pooling\n",
    "* filter 大小都是 <span style=\"color:red;\">3x3</span>, max-pooling 都用 <span style=\"color:red;\">2x2</span> 為一小區塊\n",
    "\n",
    "CNN 一個小技巧是每層的 filters 數目是越來越多。做完 convolution 之後, 我們要拉平、再送入一個標準的神經網路。這個神經網路設計是這樣:\n",
    "\n",
    "* 只有 <span style=\"color:red;\">1</span> 個隱藏層, 使用 <span style=\"color:red;\">200</span> 個神經元。"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 10,
=======
   "execution_count": 15,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Activation, Flatten\n",
    "from keras.layers import Conv2D, MaxPooling2D\n",
    "from keras.optimizers import SGD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 建構我們的神經網路\n",
    "\n",
    "一開始一樣是打開個空白的神經網路。"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 11,
=======
   "execution_count": 16,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第一個隱藏層一樣要告訴 Keras 我們輸入長什麼樣子。`padding` 設成 `same` 是每個 filter 會輸出原來 28x28 一樣大小的矩陣。"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 12,
=======
   "execution_count": 17,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Conv2D(32, (3, 3), padding='same', input_shape=(28,28,1)))\n",
    "model.add(Activation('relu'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Max-Pooling!"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 13,
=======
   "execution_count": 18,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(MaxPooling2D(pool_size=(2, 2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二次 Convolution!"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 14,
=======
   "execution_count": 19,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Conv2D(64, (3, 3), padding='same'))\n",
    "model.add(Activation('relu'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再 Max-Pooling!"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 15,
=======
   "execution_count": 20,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(MaxPooling2D(pool_size=(2, 2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第三次 Convolution!"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 16,
=======
   "execution_count": 21,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Conv2D(128, (3, 3), padding='same'))\n",
    "model.add(Activation('relu'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Max-Pooling 最終回。"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 17,
=======
   "execution_count": 22,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(MaxPooling2D(pool_size=(2, 2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然後我們要送進一般的神經網路了。記得這是要拉平的, 還在 Keras 會幫我們做!"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 18,
=======
   "execution_count": 23,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Flatten())\n",
    "model.add(Dense(200))\n",
    "model.add(Activation('relu'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "輸出和上次一樣!"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 19,
=======
   "execution_count": 24,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Dense(10))\n",
    "model.add(Activation('softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 組裝\n",
    "\n",
    "和之前比較不一樣的是我們還要做 `compile` 才正式把我們的神經網路建好。"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 20,
=======
   "execution_count": 25,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "#model.compile(loss=\"categorical_crossentropy\",\n",
    "#              optimizer=Adadelta(),\n",
    "#              metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
<<<<<<< HEAD
   "execution_count": 21,
=======
   "execution_count": 26,
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss='mse', optimizer=SGD(lr=0.05), metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 檢視我們的神經網路"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_1 (Conv2D)            (None, 28, 28, 32)        320       \n",
      "_________________________________________________________________\n",
      "activation_1 (Activation)    (None, 28, 28, 32)        0         \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 14, 14, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 14, 14, 64)        18496     \n",
      "_________________________________________________________________\n",
      "activation_2 (Activation)    (None, 14, 14, 64)        0         \n",
      "_________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2 (None, 7, 7, 64)          0         \n",
      "_________________________________________________________________\n",
      "conv2d_3 (Conv2D)            (None, 7, 7, 128)         73856     \n",
      "_________________________________________________________________\n",
      "activation_3 (Activation)    (None, 7, 7, 128)         0         \n",
      "_________________________________________________________________\n",
      "max_pooling2d_3 (MaxPooling2 (None, 3, 3, 128)         0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 1152)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 200)               230600    \n",
      "_________________________________________________________________\n",
      "activation_4 (Activation)    (None, 200)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 10)                2010      \n",
      "_________________________________________________________________\n",
      "activation_5 (Activation)    (None, 10)                0         \n",
      "=================================================================\n",
      "Total params: 325,282\n",
      "Trainable params: 325,282\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 訓練"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
<<<<<<< HEAD
      "Epoch 1/12\n",
      "60000/60000 [==============================] - 459s 8ms/step - loss: 0.1796 - acc: 0.1012\n",
      "Epoch 2/12\n",
      "60000/60000 [==============================] - 459s 8ms/step - loss: 0.1803 - acc: 0.0986\n",
      "Epoch 3/12\n",
      "60000/60000 [==============================] - 461s 8ms/step - loss: 0.1803 - acc: 0.0986\n",
      "Epoch 4/12\n",
      "60000/60000 [==============================] - 460s 8ms/step - loss: 0.1803 - acc: 0.0986\n",
      "Epoch 5/12\n",
      "60000/60000 [==============================] - 459s 8ms/step - loss: 0.1803 - acc: 0.0986\n",
      "Epoch 6/12\n",
      "60000/60000 [==============================] - 459s 8ms/step - loss: 0.1803 - acc: 0.0986\n",
      "Epoch 7/12\n",
      "60000/60000 [==============================] - 460s 8ms/step - loss: 0.1803 - acc: 0.0986\n",
      "Epoch 8/12\n",
      "60000/60000 [==============================] - 460s 8ms/step - loss: 0.1803 - acc: 0.0986\n",
      "Epoch 9/12\n",
      "60000/60000 [==============================] - 471s 8ms/step - loss: 0.1803 - acc: 0.0986\n",
      "Epoch 10/12\n",
      "60000/60000 [==============================] - 466s 8ms/step - loss: 0.1803 - acc: 0.0986\n",
      "Epoch 11/12\n",
      "60000/60000 [==============================] - 461s 8ms/step - loss: 0.1803 - acc: 0.0986\n",
      "Epoch 12/12\n",
      "60000/60000 [==============================] - 468s 8ms/step - loss: 0.1803 - acc: 0.0986\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x10aa92b00>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
=======
      "Epoch 1/12\n"
     ]
    },
    {
     "ename": "InternalError",
     "evalue": "Failed to create session.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mInternalError\u001b[0m                             Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-28-f59e3aba2b2d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx_train\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my_train\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbatch_size\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m100\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mepochs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m12\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/engine/training.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs)\u001b[0m\n\u001b[1;32m   1035\u001b[0m                                         \u001b[0minitial_epoch\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minitial_epoch\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1036\u001b[0m                                         \u001b[0msteps_per_epoch\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0msteps_per_epoch\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1037\u001b[0;31m                                         validation_steps=validation_steps)\n\u001b[0m\u001b[1;32m   1038\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1039\u001b[0m     def evaluate(self, x=None, y=None,\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/engine/training_arrays.py\u001b[0m in \u001b[0;36mfit_loop\u001b[0;34m(model, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch, steps_per_epoch, validation_steps)\u001b[0m\n\u001b[1;32m    197\u001b[0m                     \u001b[0mins_batch\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mins_batch\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mi\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtoarray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    198\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 199\u001b[0;31m                 \u001b[0mouts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mins_batch\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    200\u001b[0m                 \u001b[0mouts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mto_list\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mouts\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    201\u001b[0m                 \u001b[0;32mfor\u001b[0m \u001b[0ml\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mo\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mout_labels\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mouts\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, inputs)\u001b[0m\n\u001b[1;32m   2656\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2657\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__call__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2658\u001b[0;31m         \u001b[0;32mif\u001b[0m \u001b[0mhasattr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mget_session\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'_make_callable_from_options'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2659\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mpy_any\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mis_sparse\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2660\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mpy_any\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mis_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py\u001b[0m in \u001b[0;36mget_session\u001b[0;34m()\u001b[0m\n\u001b[1;32m    182\u001b[0m                 config = tf.ConfigProto(intra_op_parallelism_threads=num_thread,\n\u001b[1;32m    183\u001b[0m                                         allow_soft_placement=True)\n\u001b[0;32m--> 184\u001b[0;31m             \u001b[0m_SESSION\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSession\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    185\u001b[0m         \u001b[0msession\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_SESSION\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    186\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0m_MANUAL_VAR_INIT\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, target, graph, config)\u001b[0m\n\u001b[1;32m   1561\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1562\u001b[0m     \"\"\"\n\u001b[0;32m-> 1563\u001b[0;31m     \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mSession\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__init__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtarget\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgraph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mconfig\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mconfig\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1564\u001b[0m     \u001b[0;31m# NOTE(mrry): Create these on first `__enter__` to avoid a reference cycle.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1565\u001b[0m     \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_default_graph_context_manager\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, target, graph, config)\u001b[0m\n\u001b[1;32m    631\u001b[0m       \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_created_with_new_api\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    632\u001b[0m         \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 633\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_session\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf_session\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTF_NewSession\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_graph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_c_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mopts\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    634\u001b[0m         \u001b[0;31m# pylint: enable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    635\u001b[0m       \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mInternalError\u001b[0m: Failed to create session."
     ]
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
    }
   ],
   "source": [
    "model.fit(x_train, y_train, batch_size=100, epochs=12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 5. 結果測試"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1 分數"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/10000 [==============================] - 0s 35us/step\n"
     ]
    }
   ],
   "source": [
    "score = model.evaluate(x_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "測試資料的 loss: 0.0021936196734619147\n",
      "測試資料正確率: 0.9861\n"
     ]
    }
   ],
   "source": [
    "print('測試資料的 loss:', score[0])\n",
    "print('測試資料正確率:', score[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 5.2 儲存結果\n",
    "\n",
    "結果看來還不差, 所以我們把結果存起來。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_json = model.to_json()\n",
    "open('handwriting_model_cnn.json', 'w').write(model_json)\n",
    "model.save_weights('handwriting_weights_cnn.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 5.3 欣賞一下成果\n",
    "\n",
    "我們用另一個方式: 每次選 5 個顯示, 看是不是有正確辨識。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "predict = model.predict_classes(x_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAABpCAYAAAAqXNiiAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAADxBJREFUeJzt3XeslMUax/HvCCgWJFYEG8ZIBPWKXdEoUcBersZguZaIXSSol9gVrNGooGJDMaAYFRWvisaKvRMrKirKRSzYG1dQ0Ll/HJ6d3Xd38ZzD7r7vO+f3SQjLu+fszlned87zzjzzjPPeIyIi+bdU2g0QEZHaUIcuIhIJdegiIpFQhy4iEgl16CIikVCHLiISCXXoIiKRiLZDd871dM5Ncc797Jyb4Zz7Z9ptygrn3AbOufnOuQlptyVtzrnBzrmpzrnfnXPj0m5PFjjnlnHOjXXOzXLO/eqce9M5t3va7cqKLF8/UXbozrn2wAPAZGBl4FhggnOuR6oNy47rgNfTbkRGfAlcBNyadkMypD0wG9gJ6AycC0x0znVPsU1ZktnrJ8oOHdgQ6AaM9N7/6b2fArwIHJZus9LnnDsI+Al4Ku22ZIH3fpL3/j/A92m3JSu89//z3g/33v/Xe/+X934yMBPYIu22pS3r10+sHbqrcmzjRjckS5xzKwIXAKel3RbJD+dcF6AH8F7abUlTHq6fWDv06cA3wDDnXAfn3ACabh+XS7dZqbsQGOu9n512QyQfnHMdgDuA8d776Wm3J2WZv37ap92AevDeL3DO7QdcC5wOTAUmAr+n2rAUOed6A/2AzdJui+SDc24p4HbgD2Bwys1JVV6unyg7dADv/Ts0ReUAOOdeAsan16LU9QW6A5855wBWANo553p57zdPsV2SQa7pJBkLdAH28N4vSLlJaetLDq6faDt059w/gI9oGlY6EegKjEuzTSkbA9xV9O9/03SCnpBKazJiUUZUe6AdTRdoR2Ch935hui1L3Q1AT6Cf935e2o3JgFxcP7GOoUNTRstXNI2l7wL099632SEX7/1v3vs59geYC8z33n+bdttSdg4wDzgD+Neix+ek2qKUOefWBY4DegNznHNzF/05NOWmpSYv14/TBhciInGIOUIXEWlT1KGLiERCHbqISCTUoYuIREIduohIJBqdh95WUmoq1ZKpRp9JOX0mlelzKafPpIgidBGRSKhDFxGJhDp0EZFIqEMXEYmEOnQRkUhEW21RRJbcOec01SmbNm0aAGPGjAFg9dVXT61NUp0idBGRSDS62qJyRsvpMymnz6Syhn8u7dq1A2DRpg4st1zTLo4jR44EYNCgQfV4W50r5ZSHLiLSlihCr49cRBijRo0C4JRTTgHgrLPOAuDiiy9u8Wu99dZbAKy00koArLvuuskvycVnUs1ff/0FwNChQwG49tpry77m99+b9k9Zeumlm/uymY/Qk2xM/dJLLwVgp52adnmcMmVKLd8m1+dKnShCFxFpS5Tl0gb98MMPQIgybXz066+/bvZrzJ07F4D77rsPgBNOaNpa8eabbwYqRui59OeffwJw2mmnATB69GggfGYrrLBCOg1LycCBAwF47LHHAPj8888BeOSRRwDYY4890mlYM33xxRcAbLzxxoVj++yzDwC33HILAB06dGh8w2pEEbqISCRyH6FbpDh//nwAbrzxRgDOPffckq+ziCo5Z7D77rsD4bdzt27d6tfYjNhvv/0AmDlzZsnxI444YrHfN3369MLjYcOGAfDwww/XuHXZ8vLLLwNwzTXXlBzfc889Abj//vsLx9q3z/3l9Lc22WQTINyp2J3Z3nvvDYQ7mqzq1KkTABtuuGHh2IQJEwDo2LEjEP6vl1lmmVa9x6uvvgrA+PHjAVhllVUAuPDCC1v1ei2hCF1EJBLq0EVEIpHbe8S3334bCMMHn332WcnzSy1V+XeVDb0Ym9y56qqrALjiiitq2s4s+OWXXwA46qijAHj99ddLnj/wwAMB2HzzzSt+/9SpU4GQogYwb948ICw0sckwu/XOuxkzZgBhaCGpb9++QNsYZlkcuw6T11VWrbjiigA88cQThWOdO3cGYOzYsQCcccYZAKy33nqteo8RI0YA8PjjjwMwefLk1jW2FRShi4hEIhfhxYsvvlh4bFGmTeglJ2FsUYelV/Xq1QuA4cOHAyEKtckuY5Op66yzDgAnn3xy4bm8RB9JFplbZG3RlDnggAMAuPXWWwFYfvnlgTBx/OOPPwJhMYlF5cV22WUXACZOnFjTtqfNzrMXXnih5LiluA0ZMqThbcqS77//Hgjnii1Ky4t6pJv++uuvAHzyyScAnHnmmQDstttuNX+vahShi4hEIhcRejEb2zQWUffr1w+A66+/HihfHGC/PW0MzcbYLfq36NOWvQ8ePLjwvXmL0C0y33nnnYHyyNyizNtuuw2AZZddtuR5W8K+6qqrVn2PrbfeuuQ1YmGpmR988EHF5+1OL8+LT5aEpeQZuzbsjjgvPvroo8JjK+tgKcytHTu3RXbWR6Wx6EwRuohIJHIRoXfp0qXw2JaU9+jRA4B7770X+Pvfhsmkfls0UDw+D3DccccB1bNksuzRRx8FwthdMjK3sby7774bKF84YYuzqmWqbLXVVoXHliVgCzXy7ssvvwTCfIONEZtjjjkGCHMybc1vv/0GwEMPPQSE4lyW5VR8jebBG2+8UXhs1/qSXvOvvfYaEO5a0rizz1+vJSIiFWU6Ql+4cCFQOp49a9YsIIwTt3Sc6qeffgJC3rmxZb/bb7996xqbIvuZTj31VKB0iT6EyPzBBx8EqudOW/7tU089VXJ8m222AeDJJ58sHLOMmFjYmOq3335bctzuVqyQWQtK42Zacizcxo0te8z+bVvN2R3K008/DYTo0+7aVltttTq3uP6scJfNpyXnlpK+++47AN58802gtAwEhPm8/v37A7DZZpvVrrFVKEIXEYlEpiP0e+65Byhd1WWee+65Fr2Wlfe0MqizZ88ued4iD8veyDorgQshwycZmVs2i42ZV4vMn332WSAUKEuyQmexReUA77zzDhBWyyb16dMHyGdk/vPPPwNh8xEIm5nY/IrlkVvEbXMINkZ+9tlnA3DXXXeVfP0aa6xRcjxvKq2KfvfddwHYYIMNADjvvPOAME9kpYJtzYqNEti1mBwzX3/99SserydF6CIikch0hL44lt1iv0WN1Sn5448/gFDCcty4cUD5ylKrQWJlQfPCfi4ojcAgrAC1HPFqZUA//fRTAPbdd18gZDJYjrWNp1o0FiNbd5DMarGVwhbR5omNj9vfl112WeE5WwFrmVBz5swBYNNNNwXgpZdeAsKWcraKOBllXn311UAYY88by5IDuO666wC46KKLAPjqq6+AUBo4eRdj1l57bQAWLFgAhLUulvnzwAMPAI3NR1eELiISiUxvEm0rFm3j4eJjJpn/+s033zS9UTN/rvfeew8oLXhfA3Xb5NbGLIs3o7AIwcaBk7VZkmwW//jjjwfg9ttvL3neInSrSWE5u8WbWVx++eVAqFTXDJna+Nd+pgEDBgBhHNTOJzsvVl555Xo2oy6bRNtciN1dHH300YXnRo4cudjvtXH3nj17AiGCt+jUInmL/uu0YjaVc8WuC5tfeOWVV5reYFFfst1225V8/RZbbAHAlVdeCYSaR7aWxSL/GtEm0SIibUmmI3RTPEZ8+OGHAyGCSmrXrh0Q6phYJHvssccCYZzYIjGro17jutZ1izAWtwrtww8/BMLsum3Pl2Rjg3feeWez3tPq5VitCgjRSQtkIkK3PHOLQJORuWU71fiOrZq6ROh27hsbD28O237N1jRY/2BRvkX/gwYNAmDMmDHNfu0WyMS50lxHHnkkELayGzVqFFC6fqYGFKGLiLQluchy6d27d+Gxjd1ZbvozzzwDwCGHHAKESMtmoG08LDn2fthhhwFx7Thjuy3ZrPrfjZcm2Wdh0ZjVOrddoezuJ4/s///0008HSvP4Idy1NCgyr6uuXbsC4Q6seM3GjjvuWPK1Nidz6KGHAmGM3CLzm266CQjnxEYbbQSE8XnL6LCotC2yOSi7a7YNxNOgCF1EJBK5GENfEpZjbfv62fiy5au3IEujJVIZQ2+tbt26AaFanP27xlIZF7XI3DJ6ivP3IUSatjKywStC6zKGbquirQ5N8bli9XzsztVq1Fh2mOWVW555cgWtZVSddNJJQLhDtnkui9iXUK7G0K1Ko33OVg+9tXXVq9AYuohIWxJthG5jpFa/2rIbLFKrcY5oUt0iDMufvuSSSwrHJk2a1JKXKOjevTsQ9s2sU2RuGhp12Uphy25K7qy0ww47AKGyZEo7ENUlQjc232TV/iCsZrSoMrkK0laKWoXNv2Pj9RbhJ1dLtlIuInSrsrjlllsC4TP8+OOPAUXoIiKyBOJJ8Uiw2flkfWvL0W1uzeOssSpxd9xxR+GY5dYn2c+41lprVXzexlrrHJk3jFXLgzB+bOsMjFXTtJ895r1BLcq2Fb8QctKnTZsGhOjdoknLDmsum3fZdtttAdhrr72AcP1BqF4YG1v3kSWK0EVEIhFdhG77Yp5//vkVnx8xYgQQdvHJq+IKitWqKVaL3K3mhGX85J3tbGV126E8Mh8yZAgQ5h5qlI2RC8W7CQ0cOLDk7yVlEb1lw1j9oOJ8fqvTH2ukniWK0EVEIqEOXUQkEtENuZjkRhZm6NChQJzbqSXdcMMNFY8PGzYMiGdC0Cb4bNFMsYMPPhgIP3NbGmppJFv6b9sZvv/++2k2pyEs5dP+3nXXXYGapyu2iCJ0EZFIRBehFy+4KTZ8+HAgFPGS/LPFQ5UWwZx44olAWMKe58JieWKbTNvfMUuW4bDFWmlKvwUiIlIT0UTotmTZNv01nTp1AuCggw4CoGPHjo1tWIr2339/oPpdS96NHj0aCAWjrKwDhKJTtSxiJpJ1itBFRCIRTXEuy16xyMw8//zzAPTp06deb11JLooLNZg+k3J1Lc6VY7k4V6w08wUXXACEu//iBW41pOJcIiJtSTQRum1gYRtarLnmmkAo2FSnjSyqyUWE0WD6TMopQq9M50o5RegiIm1JNBF6xijCKKfPpJwi9Mp0rpRThC4i0pY0OkIXEZE6UYQuIhIJdegiIpFQhy4iEgl16CIikVCHLiISCXXoIiKRUIcuIhIJdegiIpFQhy4iEgl16CIikVCHLiISCXXoIiKRUIcuIhIJdegiIpFQhy4iEgl16CIikVCHLiISCXXoIiKRUIcuIhIJdegiIpFQhy4iEgl16CIikVCHLiISif8DGX3pFGqLwI4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 5 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pick = np.random.randint(1,9999, 5)\n",
    "\n",
    "for i in range(5):\n",
    "    plt.subplot(1,5,i+1)\n",
    "    plt.imshow(x_test[pick[i]].reshape(28,28), cmap='Greys')\n",
    "    plt.title(predict[pick[i]])\n",
    "    plt.axis(\"off\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.4 小結論\n",
    "\n",
    "我們到此, 基本上是「亂做」的神經網路。有些同學在不斷試驗的過程中, 可能會發現有時會出現很糟糕的結果。因此, 接下來我們要介紹怎麼樣用些簡單的手法, 能讓學習效果比較穩定, 而且有可能可以增加學習效率。"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
<<<<<<< HEAD
   "version": "3.6.5"
=======
   "version": "3.6.6"
>>>>>>> 54608a95a9c2e225243a1e873f0d9d03f08899d6
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}