vanishing_gradient.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "**The vanishing gradient problem occurs when the gradients of the loss function with respect to the network's weights become extremely small during backpropagation. This makes it difficult for the model to update the weights effectively, especially in the earlier layers, leading to slow learning or the model failing to learn entirely. This issue is common in deep neural networks with many layers and activation functions like sigmoid or tanh, which squash input values into a small range.**\n",
        "\n",
        "# **VGP happens mostly with sigmoid and tanh activation functions**\n",
        "**REASON:**\n",
        "* Sigmoid maps the values strictly between 0 and 1\n",
        "* tanh maps the values strictly between -1 and 1\n",
        "\n",
        "\n",
        "# **TO DETECT:**\n",
        "* Recoginize loss for each epoch\n",
        "* plot value vs epoch graph, to check if the weights are changing or not (We will do this in tensorboard)\n",
        "\n",
        "# **FIXING VANISHING GRADIENT PROBLEM**\n",
        "* if we know there is not that much non linearity to be captured, we can reduce the size of the model architecture by removing hidden layers\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "\n",
        "* Using other activation fun. like relu (it does not squish the values as much as sigmoid and tanh) however it still squish the -ive values to 0 which is a down side.\n",
        "* The gradient of -ive is 0 while 1 for +ive values in relu, so by multiplying many one's it does not gets smaller, thus prevents VGP.\n",
        "* If many zero's than it can cause 'dying relu', to solve this we can than use 'leaky relu'.\n",
        "\n",
        "FORMULA: max(0, z)\n",
        "\n",
        "\n",
        "---\n",
        "\n",
        "* Proper weight initialization like Glorat/Xavier & He\n",
        "\n",
        "---\n",
        "* Using Batch Normalization Layer\n",
        "\n",
        "---\n",
        "* Using Residual Network\n",
        "\n"
      ],
      "metadata": {
        "id": "AsSz55d3WGHs"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Exploding Gradient Problem in RNN's (Reverse of VGP)**\n",
        "**The exploding gradient problem happens when the gradients of the loss function become excessively large during backpropagation. This can cause the model's weights to update with excessively large values, leading to instability and divergence in the training process. This issue is typically encountered in deep networks or recurrent neural networks and can be mitigated by techniques such as gradient clipping, where gradients are scaled down to a manageable size.**"
      ],
      "metadata": {
        "id": "FNjWdVmycMEL"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IFGFeZeCSUIX"
      },
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import tensorflow as tf\n",
        "import keras\n",
        "from sklearn.datasets import make_moons\n",
        "from sklearn.model_selection import train_test_split\n",
        "from keras.layers import Dense\n",
        "from keras.models import Sequential"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "X,y = make_moons(n_samples=250, noise=0.05, random_state=42)"
      ],
      "metadata": {
        "id": "VNiEr_mpSkTO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "plt.scatter(X[:,0],X[:,1], c=y, s=100)\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "UVpq7UCcTTA6",
        "outputId": "0ccd48a9-4d6a-4aa7-8249-8501d0132444"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3gURRvAf7PX75KQ0BGkKIigdBAUUURBEMEGiFhQUGxg+RSxoygKNlSwg72gglRRUFFUmoAU6b33mnZ95/vjQkjIXXItgeTm9zx5yO3OzrxH7vbdeauQUqJQKBSKxEU71QIoFAqF4tSiFIFCoVAkOEoRKBQKRYKjFIFCoVAkOEoRKBQKRYJjPNUCREPFihVl7dq1T7UYCoVCUapYsmTJQSllpZOPl0pFULt2bRYvXnyqxVAoFIpShRBiW7DjyjSkUCgUCY5SBAqFQpHglErTkOLUsmfzPg7sPIQ92UadxjUxGAynWiSFQhEDShEowmbJL8sZ+/iXbF+zC5PFhO7XsdjN9Hr0Gm7439VoWunZYG5dtYNVc9ei65Kzm9SiQZtzEEKcarEUilOCUgSKsJj45nQ+GvIlfq8fAI/LC4Az08Vnz33HyrlrGTrx0aDKwOf1cXjvUTSDRvmqqVErDL/fz7+/rGDH2t0YzUaaXX4+Z9avHtEcW1ftYORto9m+dhdCABKEJkirksojY++lSfvzopJNoSjNiNJYdK5ly5ZSRQ2VDFnHsnjxxlEsnrW80HFWh4V737idq+66IvfY0QPHGD9yMj999Bt+vx8pwZFi54aHu3LdA1dhtpoLzOP3+1n4478smL4Ej8vDWY1r0alve5bOXsk7gz7G4/bi83jRNA0JnN2kFk989SDV6lQp8r1sXbWDBy56Clemk2Afe4vNzLApQ2h+ReMi51IoSiNCiCVSypYFjitFoAiFM8vFoDZPsmPtLnS/XuT4amdV4bMNoxFCsH/HQQa2foKMw5n4PL584yw2MzUbVOf1OcOwOay5x1cvWM/Qa0bidnlwZrgAMNvM+Dw+hBD4ff6g61ocZkb8/DQN2pxTqL/ivpaPsXHplqBK4DjlKibz7Z6PlN9DUSYJpQhKj1FXUeJMGf0TezbvC0sJABzYeYgJb0xj9td/8WSX4Rw7kF5ACQC4nR62rd7J2/d/hCvbja7rbFq+lSEdh3H0QHquEgDwOD3ofj2kEgBwZ3l4uN2zXJ10Cy/c+AZLZ6/k5Aecrat2sH3NrkKVAIDH7WXh9H/Der8KRVlB7QgUQdF1nRvPGMDR/ceKdR1NE+hSIoRA6vH5LAohqHBGGve83pdLe10EwIyPfuXdhz/Bne0p8vob/teNe167LS6yKBSnE6F2BMpZrAjKsQPpZKdnF/s6es7NP54PJFJKDu46zKv93uHIvqNcO+iqwDphLqH7Q+8+gnFozxGmfzCLvyYuwOvycua51bnh4atpetn5KhJJUSpQikARFCll2Cah0xV3tocPH/uC1le34OymtRFa0TdlW5KVc1qcHfYav371J6MGfABS5kZS7d60j+VzVnN2k9q8NONJ7Mm2qN+DQlESKB+BIijlKqUUaU8vDXjdPgY0foQVf64mrUpq0RcIuKRHm7Dm/vfXFbw54AM8Tk+uEjiOK9PF+sUbefbakdGIrVCUKHFRBEKIj4UQ+4UQK0OcF0KIt4UQG4UQK4QQzfOc6yuE2JDz0zce8ihix2Aw4ChnP9VixAVXlpsvnvsOo9mIxVYwZPU4FruZgaP7Fwhr9Xl9HNx9mKMHjuUzYX04+AvcztA+B6/bx9qFG1m3eFPsb0KhKEbiZRr6FBgDfB7ifBegXs5Pa+A9oLUQojwwFGhJwIK7RAgxVUp5JE5yKWKget2qpB/KONVixAW308OeTXtpdkUj1i/ahMflxZkZiE6yJQVCWAeO6U+n29rnXnNk31G+fWUyM8b+hu7T0XWdtMqp9BzcnaYdGrFz/e4i1/W6PEx792fqf3x/sbwvhSIexEURSCn/FELULmTINcDnMvA4tUAIkSqEqAa0B36RUh4GEEL8AnQGvomHXIrYuPqeTmxZuR1XlvtUixIXfF4/i35axruLRnBg12H++2sNul/nnBZn0+6G1vl2Anu27GNQmyfJOpqFz3vCebx/x0HGPv4lZ5xdFYPJAM7C19R1yc4Ne4rrLSkUcaGknMXVgR15Xu/MORbqeAGEEAOAAQA1a9YsHikV+bi014V8MPjzuCoCITjlvodHrxjG11vf5aLurYKel1Ly6GXPcexgetBII3e2JycnITxnunIWK053So2zWEr5oZSypZSyZaVKBRrsKMLE7/Mzb+oivnpxIt+8PImVc9eGDN202Cy8+utQHKl2TObYnxnMNhPJ5ZM41RGVrkwXMz/9I+g5Z5aLgRc8zv7tBwsNN/X7/Oj+ojWaLcnK5TdfEqWkCkXJUFI7gl3AmXle18g5touAeSjv8T9KSKaEY87383nr3g/xeX24stwIITBbTaRVSeXpbx8OGjZ5VuNajF05iklvzWD6B7NwZbpyY/8jpf2NF9F7yHU83O4ZnFkuPE5v0RcFweqw4PfrNG7XgB3r93Bg+4GIdhm6X+eHt37k+ge75jsupWToNSPZuGxLVHIFw2gy0C7MKCSF4lRRUjuCqcBtOdFDbYBjUso9wEygkxAiTQiRBnTKOaaIM7+Pn8urd4wh43AmzgwXUg/kCbiy3OzZvI9H2g9l49LgN8CKZ5TnrpG3MOXo50zL/JJXZw/F6rBEtL7BaODmp3twZv3qfLp+NHe8cBOVa1bEYIzsI2ixW2h77QV8vfVdRsx8hq+2vMvP3m8Z9M6dEc1zeM/RAsdWzVvHmoUbwnrSLwqhCaxJVl788UnMFlPM8ykUxUm8wke/AeYD9YUQO4UQ/YUQ9wgh7skZMgPYDGwEPgLuA8hxEr8ALMr5GXbccayIHx6Xh1F3v19oeQVXlpvX+r9b5Fxmq5mm7c+n0+3tI1IGJosxt/x0UqqDHv/rxldb3+Nnz7dMz/qSK+9oj7mQ0M7juLPdzJ30D/e1epz9Ow4CoGkaHW+9BCIwORnNBYvKTXp7RlglKArDardgsppod0Mb3l00goZtzolpPoWiJIhX1NBNRZyXQND4OSnlx8DH8ZBDEZw5388Pq7zCpmVbuaFSP85uWpvq9apxTouzaH/jRdiSCjo7B77dnxr1zuCjIV/gdRcsLHcyBqOBitXLBz1nsVn430f3YraamfnpH/i9/kKLzLmy3XjcXgZ3eI6P176FwWDAlmSjWp0q7Nm8r+g3SqBS6slsX70zplIXJouJ6x/uSq/B1+BIKRs5GIrEoNQ4ixXRs2ru2tyY+aJIP5TB0t/+Y/r7s3jnwY/pWfUuvn9jaoEbpBCC6x64ijfmDMNUhOnDZDFy9T0dMZpCP3domsYD79zFu4tH0qV/hyLn1P06R/YfY/HPy3KP9R5ybRjvMMDBnYcKHDPbYjXhSMpVTFFKQFHqUIpAERJ3tgd3tpuxj3/Fw5c8w9+TFhZ4Uj/3gnpc3udirPbgZiKDyUBa1VRufCy8m3StBjXo+Wh3tDDqAjkzXEx7f1bu6yvvuCxs81DG4Sx0PX/45yU9Liw087goDEYDtc87s+iBCsVphlIECUCjdg1zs2ejQffprJq7jlf6jqFn1Tv556el+c4//NE9dB/YGYvNnOs3MFmMmK0mzm97LmMWjiA5LSns9Y7sO4YxzHDVg7tOuJQMRkPQrmfB0AxagZaZXfpfHraMwXBlu0k/XDYysRWJhVIECUC7G1rHpRyyM9NFxuFMhvV4LV/rSk3TuGvELXy75yPuf6sftw3txR0v9uHDFa/z2uznSKtcLqJ1kssn5fZGLopylVLyvW7ZqUlY77XZ5Y0KHEupkMzAMf3DEzIYEob3fpPeNQbw21d/Fdhx5MXv9zN38j88ePHTXFu+L9dXuoPnb3iN1fPXRb++QhElqjFNgjDn+/m8evuYQoukRUKlGhX4att7xVJvX0rJbWcPZO/W/YWOsyVZeWTsvbnNZyAQAjqk0wu4s0NnQ1sdFl6Y+jhNLzs/3/Hdm/Yy9LpX2LpyR4grw8diN9PyyqY8893/CrS9dGa5eOLKF9m0fBuurBO+GyEEZpuZzv0u4/63+qleBoq4o1pVlkL2bTvAJ09/w3PXv8rIvqOZN2URfp8fv9/Pop+XMnnMT8wY+xv7th0ocq5Le17IE189SLmKKXFwikLG0SxWzFkd8zzBEEJwy7M9sITwOwTGgDXJykXX5i8Tcd5F9en5SDcs9uAmIovdQvf7riygBNYv2cQ9zQfHRQlAwL+yeOZyvnl5UoFzL/d5iw3/bs6nBCCgAN3ZbmZ+/Dvfvz4tLnIoFOGgdgSnIT6vjzcGvM8f4+chpczt+2tLtiKEQAiBruv4vX6EpqHrOue3PZfHvxhE+appQeeUUuLMdOF2efj967947+HPYpLRZDFy54hbCmTnxgspJe89/Ck/jf0N10lP9yaLEavDyqi/XqBWgxpBr//j27l88sx4Du0+jNFkxO/1k1qlHLc914uOt16ab6zH7eWmGgNIP5QZ9/eRlOrg+31jcyOmdm3cw4DGjxToX1DUdQpFPFCtKk8z9m7dz4YlmwGo1+IsqtaunHvuldvHMG/yIrzu/DeLvE3dT2bFnFXc22II7y99NZ9N3u/zM+uzPxg/cjL7th5A0wQGsxGhxdojWBRwtsYTIQT3vXkHrTo3ZfyIyfz39xoEYE+x0+3eTlz3wFWFNpppf2NbLu11ETvW7ebYgXRSKiZT89zqQc0tf/+wMKxciGjQdZ3V89fT+JKGAPz6xRz8YXR+03Wdf3/9jwu6NCsWuRSKvChFUMJsX7uLt+79kLULN+RGxvg8PupfUJeH3r8bv9fHvCmLIrbl+306xw6m89FjX/DYpwMB8Hq8PNX1JdbM35D7VO0HvJ7Yb3qaQXBe2/oxz1MUrTo3o1XnZui6js/jCzsqCALKpOa51eHcoAVtc/n1izlh51lEjITMo1m5L/duOxiWI1z36xzeW7AMhkJRHChFUIJsXbWDB9s+Faj1k6fHLcDKv9YwqM0TtOjYOOqnU7/Xz5zv5nP/2/1wpNgZ98RXrJ63Pm4O4rxUrV2Zes3Pivu8odA0LSIlEAlZx7KLZV6A7Awn7zz4CT9+8Aud+3UgpbwjrN2YZtBISlWJaYqSQSmCEuTFG9/AmeEMWilTSnBmOFkw/d+YmsabLEa2/Ledus3qMO29WUXaoqPBlmTl8S8fiPu8p4oz6lZlzcINMZrKQrN/2wH2bzvAf3+vwWw1YzQb8Rbxd/H7dFp0alIs8igUJ6OihkqI9Us2sXdr4eWSpaSAXyAadL/Oc9e/GpMSMFlMVDqzAlaHBYvdnPtzVpNajPrrBeo2rROznKcL3e69MqaM4nBxZrhIP5iO3+tHM4T+6llsZrr074DNEX0SoEIRCWpHUEIsm70Sn7d4HJJ5cWW7WfjjEpbNXhn2NZpBw+qwIITA6/ZSv1Vd7nmjL+e0OJt1izexYclmNE3QoE096jSqVYzSnxoatK7H2U1rs37Rprj4TwpDSjAYNEyWQCTTycra6rBQv1VdBrx6a7HKoVDkRSmCEsLn9Reb6SEvUpdMfXdmodU7T+a6B6/iou6t8Hl81DinGpVrnugAV7/l2dRvWbBhTVlCCMHw6U/wWMcX2LF2V2SOY0FYlV3zEnB6m+j9xHVMHv0T2elOpK5TpXZleg+5lk5922MwFiyTrVAUFyqPoISYN2URI24bjTOjiG7nMWI0GxFEFhn0xpznadSuYfEJVUrw+/0snrmciaOms/LvtUWa6WxJgbyO7Cj+pgajgUlHPsVqt5Cdns3uTfuY8dGvLJ29EqnrnNu6Hjc8fHXQrnEKRbSoPIJTTOuuzSPuxhUNPo8vogYtAGv/2aQUAWAwGGh9VXNaX9WclX+v4fErXwwZcSWEwJZso+21rfhp7G/4wqyNdBxd13P7QI9/ZQo/vPljvj4Me7fsZ+7kRVx83QUM/vT+AmUqFIp4opzFJYTBaGDg6P4hSx+cSg7uKlibP9E5/+IGPPTBACw2MwZT/puw1WEhtXIKb8x5nh7/61bgfDg0aF0Po8nIpLd/ZNJbM/A4PfnMeboeKDfx96SFvPfwp7G+HYWiUJQiKEEu692WAa/eitlqytfm8bij9lQgDKJABU9FgCtuuZT3l71G1wEdSa2Ugi3JSvV61bhzxM18un401etW44yzq/LIuPuw2MyE+ye0OizcOORavB4vnz/3faEF8tzZHmZ89BvHDqbnO55+KIN5Uxcx5/v5bF6xLZa3qVAo01BJsH7JJsaPmMS8qYuRusRgMnBW41okpTqwJdtofElDbMlW3r7vI1xZoW8K4VK9XjV2b9gbVttFo9HApT0vjHnNskqNetUYNLo/g0aHLk992Y1tqVq7Ml8O+55/f/sPKWXI7GGL3UKHm9txYbeWzJ+2OKwAAqEJfv9mLtcO6kL6oQxGDxzLvCmLApnpEvx+nSq1KzHw7X4061CwvLZCURRKERQzP38ymzEDx+Fxe3O/9LpfZ+0/GzFbTTw74VFaXdkUv8/P9PdnsXHplpiTwA7uPES5yikc3Xes0HEGo0ajdg2pXrdaTOspAqae4T8+SXaGk/RDGfz352q+eXkS+3ccwmgy4Pf7SUp1cPPTPeh61xUIIdi//WBYIcUep4fdm/eSfjiD+1oO4dCew/g8+UNPt6/eyTPdRvDkNw9xUfdWhcymUBRERQ0VI+sWb+KRS58ttMSD1WFh3Oo3qXxmRZxZLkbc8jaLZy7D79fDbs4SDJPFxNX3XMHkt38KmsRmMhupdnYV3vz7xYi6hykiY/vaXRzZe5TsjGyO7D/GhsWb8ft1zqx/BgajgU+e/qbIXaCmCfo8fQP7th7g9/Fzc6vRBsPqsPDd3rEqGU0RlGKNGhJCdAbeAgzAWCnliJPOjwIuy3lpBypLKVNzzvmB/3LObZdSdo+HTKcD41+eVOTTvd/rZ8qYn7lr5C3YHFaen/QYe7bs47ev/mLauzM5vO9oxHHqEKjX7yjn4Efn13z90g9Mf28WRw+kIzRB+Wqp3Dj4GrrceUXIXsOK2NF1neV/rOTLYRMK/B1NlpyCg2Eoe5PVRPPLG/H4lS8WqgSO89uXf3H13R2jlluReMS8IxBCGID1QEdgJ7AIuElKGbRriRBiENBMStkv53WmlDKiR9LSsCPQdZ2rrH3CSuwqVymF7/eOLeAwPrDzEPe1HELG4Qz8vsjrDzVpfx6vzX4u97XXEzBPFVfxNsUJdF3npT5vsWDqYtyu0DtCzaAhpSzUV1C1dmUe+mAAL/R8g6z0ogvktb66BS9OfTwquRVlm+LsUHYBsFFKuVlK6QHGA9cUMv4m4Js4rHtaU1gkyMkcO5BOn1r38MNbP+LJk8RUqUYF3l/6Kpf0uBCz1YSjnD1ftFGkmMwmpQRKiJ/HzWbhj0sKVQIQ8BchZaF5Akf2HWXCqOnIMLeGvmLqraAou8TDNFQdyNvfbyfQOthAIUQtoA4wO89hqxBiMeADRkgpJ4e4dgAwAKBmzZpxELt4sdgtaAaBP8zv5MGdh/n4qa/5ffxcXps9FIstcMOvUC2NJ79+iIwjmWxevo3Mo1kMv+nNIrNezVYTzS4/v9AxiuJBSsk3IyaFHwEmBFKG3vG5nR5WzFmNPwzHsslspF6LslMQUFEylHQeQW9ggpQyr72kVs5WpQ/wphAiaE69lPJDKWVLKWXLSpUqBRtyWqFpGu1vbFtolcmTcWd72Lx8K+8+9GmBc8lpSTRpfx5tr72AC7u3DGveq+5SduJTwf7tByNqKiP1op/1PU4PYSUqaIJu93QKe22FAuKjCHYBZ+Z5XSPnWDB6c5JZSEq5K+ffzcAfQJnpzXfjkGtznYLh4nF5+fXLPwu1Bd/7Rl+SUh1oWvAbg8Vu4fYXe+drWakoOVzZ7oiLxoWTT1BUFJnFbuGqOy/PVzRQoQiHeCiCRUA9IUQdIYSZwM1+6smDhBDnAmnA/DzH0oQQlpzfKwJtgaBO5tJIrQY1eOa7R7DaLRGVIfD7/Pzz09KQ5ytWr8A7i0ZQr+XZmG1mTBYjBqOGLdmKo5yde97oS8//lZngq1JHhWppYUX35BKHpHKTxUiX/h247807Yp9MkXDE7COQUvqEEAOBmQTCRz+WUq4SQgwDFkspjyuF3sB4mT9MqQHwgRBCJ6CURoSKNiqttLyyCTc9eR3fvjKFbG94VSr9Xj9T3/2Zy25sG3JM1dqVGbPgZbat2cmSWcvxun2cUbcqba5ujslsipf4iihISnXQ8somLJi2uNBGRLnEmMqjGTS6DujI/W/1i20iRcKiEsqKEb/PzzPXjOS/Oatzm8eHi9Fs5KP/3qBGPZX1WxrZvGIbD1z0VFjRY+H0MC6KpFQHkw5/GtMcirJPcYaPJiRSSlbOXcuEN6bx/evTWP7HqgK1fb4ZMZkVc1ZFrARyFmDqOz/HSVpFSXNW41oMmzIEW5K1yDaY8WhYlJ2enS/0WKGIBFVrKApWz1/Hy7e8zdH96fi8XpCBkg5JqQ4e+2wgTS87H7/Pz8RR03BnFx5HHgqf18/6JZvjLLmiJGl+eSO+2fE+sz6fw69f/Enm0SysDgtnnludeZMXxaU/9XEk4Ha6+XviAtIPZVKuUgqtuzbHnmyL2xqKsotSBBGyev46Huv4QoEtv8/rx5np4umrX2bY1MexOizoUWQD58UYRZ17xemFo5yD6wZdxXWDrso99mLvN+LevzqtSiq9z7gbg0nD5/FjNBvQfTpdB1zBgFdvU60vFYWiFEERSCn57681THhjGhuXbuHwniOFlntwOz2MuPVtBn9yPyJEeGc4mK1m2nRrEfX1itOXdYs2xbV/taYJMg5lBNqT5rRbPr7b+PGjX9m9aR/PT34MTVOWYEVwlCIoBK/Hy7Aer7Ps95W4s93hRYAArkwX+3ccjKjMxMn4vD4639Eh6usVpy+h8j+iQRgEQoiQPard2R6W/r6SeVMWcfF1QRP+FQrlLC6M1/u/x9Lf/sOVFb4SAHBmudixdhe+MArOhcJqt5CU6oj6esXpS9MOjeLWv9pitRRZkNCd5eabEZPisp6ibKIUQQj27zjIXxMXFNpLICQS1i/ehDEGu2yszWkUpy/XP9QVgyn2zbjQBK4sV1hj1y/aFPN6irKLUgQh+OWzP8Jq9RgMW5KVYwczwqo1Hwp7ior2KKvUalCD6x+8KmQvCKPZEFa2sdEcmTJZs2B9ROMViYNSBCHYvXkf3ijL+ZqtZpJS7VGvbTQb6XjbJVFfrzj96Te8D3cM742jnB17sg2z1YQ1yYrJauKi7q1oe+0FmCyhM8QNRo1KNSpEtObnw76PVWxFGUU5i0OQnJYUVcanxWZm8Kf3s2ruWjYs2RJVrLjRaODaPOGGirKHEILrH7ya7vd1ZtHPy9i//SC2JCutujQjrXI5stKzebDt0+zZvC9QeTQPRrORchWT6TX4Gt6854OwS1Ssma92BIrgKEUQgkt6XsiPH/4Sdk15W7IVs9XM4E/up/VVzTmrcS0mjJoe0ZoGo4bRbOLZCY9QtXblaMRWlDKMJiMXdiuQ8Y8jxc6YhS8zcdQ0fnhrBu4sN0ITaJpG17uv4MbHruXo/mMIIcI2YYbb2EaReChFEIIGretRpVYldqzdhR5iV2A0G6nVsAZtr72Ac1qcRcvOTXM7TVWqUYHeQ67l+9emBlcmInATEELgdXuxOixcfvMl9Hikm6ovpAACkWM3P9WD3o9fx6HdR/D7/BzafRhnhov92w9St1kdqtapzJ5N+8Kar1L1yExJisRBKYIQCCF4cfoTDGrzJJlHMgs4fi02MzUbVOeNP18I6fS79dmeWB1Wvnj+O4QQODNdGIwGjGYD1etWY+jERznj7Kr4/f5CWxUqEhtN05g3ZRFfD/8BV7YLTdPQ/TrJaUlcdnNbvh0xJax58oaZ7tt2gOkfzGLz8m2YrSbaXteaS3q0Ua1MExRVfbQIjuw7yviRk/lp7G+5TcbNVhNNOzSiTdfmtLiyKRWqpRU6hyvbzV8TF7B7014sVjMtr2xK3WaqneDJSOkB7zLQs8BQDYz1EeF05SrDSCkZdfcH/P7N30F3lha7hbQq5di7ZX+Rc5ksRsauHMV3r01h1mdzkLrM7ZtgS7KiGTSGTnyUZh0axf19KE4PQlUfVYogTLweLzM/+Z3Pn/8eZ0agr4AQAp/XT/OOjXno/QGsXbiBhTP+xePycnaT2lx5e3vKVUwpUTlLI1J6kJmjIfurPAf9YKiCSH4cYU3cDOuFPy7hxd6jCvVVhRvU4Chnp+FF9Vn+x6oCDujjWOwWXps9lHMvqBe1zIrTF6UIYmTW53/w9r0fBU0w0wwaUkosdguuzECCj8VmRkpJj0e6cfuw3gn/ZBsKKb3Iw/3Au5zcQjn5sELyE2iOm0patNOChy5+mlXz1sVlLrPNjNflLdK5fG7reoye/1Jc1lScXqh+BDGQcSSTt0IoAQDdryN1masEIFB8zuPy8sObPzLuya9LStRSh8z+shAlQOB4xktI/+6SFOu0QNd1VscxCczj9IQVYbR24QZmfvp73NZVxAcpPUgZfZJqYShFEAYzP/mdaB/oXVlufnjzRw7tORJfocoAUuqQNY7QSiB3JDKv2ShB0P16zG0so+Wtez9k5d9rTs3iilykfhQ9YzT6vtbIfY2R+xqiH+qBdM2KuvJBMJQiCIN/floadYOZAJJp76luYwXQ94CeHsZAD7h+LXZxTjeMJiMpFZPDGlvhjLSQndAMJkPEpkmv28foQeMiukYRX6RvJ/LgVZD1IcgjgA5I8K5AHhuMPDYkbspAKYIw0P2xNZjxun18PXwST3d7mTULN8RJqjKA9III8yMoE7MI33UPdMFsDV1qAsDqsHDrsz15+eenqdusNhabGXs5O/YUGxabmVoNa0SVTLZrwx62rtoRreiKGJBSRx65HfTDQJBAAekE10xk9idxWU8pgjCo1zz2UE8pJf/M+JfBHZ7j1y/nxEGqMoBWORAdFA7Gs4pXltOUbvdeiSPVEbKHgcGokVYllctvuTTR1L8AACAASURBVIRG7Rrw3pJXee/fV3j880E8890jfLd3LM2vaByViclgNLBt9c4Y34EiKjzzQT9EYBcQCidkfhAXv0FcFIEQorMQYp0QYqMQ4vEg528XQhwQQizL+bkzz7m+QogNOT994yFPvOnU97K4zCNlwIn85t0fsm21etISmh2sXSjyYyjsCEe/EpHpdCOlfDJvzxtOldqVsSVbc48LAdYkKzUb1ODNv/MnNZ5ZvzoXdmtJy05NsCfbqNWgBlZH8KTHojBFWOFUER+kcyLIrDBGenKCLWIj5r+yEMIAvAN0BHYCi4QQU6WUq08a+q2UcuBJ15YHhgItCTyzLMm59rTyrNY6rwYGkwF/DGWl8+L1+Jjw+jQeGXdfXOYrrUipg+l8cE0tZJQZjA3BfGGJyXW6UbV2ZT5d/zZLflnBzI9nc3jfUSrVqMhVd11O40saFmn/v7TXhYx54OOI1/W6fTS86JxoxVbEgn4ozIEC9GMxLxcPdX8BsFFKuRlACDEeuAY4WREE40rgFynl4ZxrfwE6A9/EQa64oWkaV9/dkanvzoxLr1ndr/PnhAUJrQik9CGP3AveRRS+/dXA0ilgExXRl/Yu7WiaRqsrm9LqyqYRX2tLsnHrsz34YtiEsNunGkwGWndtTmqlchGvp4gDhqoEmlIUdb/RwVAx5uXiYRqqDuS1c+zMOXYyNwghVgghJgghzozwWoQQA4QQi4UQiw8cOBAHsSOj16PdsSVZix4YJm5n9P2MywIy41XwLASZXcRIF2SOQh7sivSHV1xNUZBeg6/hxse6Y7aainQ+G0wGUiul8MC7d5WQdImL1I+iZ45DP3g9+oGr0I/cj/T8A9ZeIMK434gUMJ4fsxwl5SyeBtSWUjYGfgE+i3QCKeWHUsqWUsqWlSpViruARVG5ZiVe//35uM2XXD4pbnOVNqSeDdnjKTp/4DhO0PciD98WMCcpIuLw3iN8/vx3/Pzx7xjNxsCNvkoKVWpXIrVKOcxWE/YUG/aUQIOci7q35L1/XyWtstoNFCfSNRu5/1LIfAt8K8G/Edy/Io8MgIxXwFCLwo02BnAMjEvVgniYhnYBZ+Z5XSPnWC5SyrwGr7HAK3mubX/StX/EQaZioW6zOtiSrTgzwr2BBcdkMdH17k5xkqp0IfVMZPpwgobEFYof9P3gmQeWi4tDtDLJv7+uYOh1r6D79Xx9sI9/hk0WI82vaEz7XhdhL2en4YXnKHNQCSA9S5FHH6Lgw5AM7JJ9y4BUijQNef5A2nvGrAzisSNYBNQTQtQRQpiB3kA+758QIm+B/e7A8ZTFmUAnIUSaECIN6JRz7LSl6WWxb8PMVhPd7008RSDdC5AH2uU4h6N4spdZyOzTyn10yvF6vKxbvImVf68pkL2+Y90unr32FVxZ7nxKIN/1bh///rqC8SMn06zD+UoJlBAy41UK3xFL4AhQWICKH9zzAqGmMRLzjkBK6RNCDCRwAzcAH0spVwkhhgGLpZRTgQeEEN0BH3AYuD3n2sNCiBcIKBOAYccdx6cLh/ce4adxv7Fx6VbMVhN1m9VhwfQlUTmNLXYzRrORkbOepXzVwktXlzWkdz3yyN2AM7aJlJ8AAI/Lw5cvTDgRwCACocnJaUnUOq8GLTs2YdOKbWG1SvW6fezZvI9vXp5Ev+F9SkD6xEb694J3RZxmy0ZmjUVYLoppFlV9NAS6rjPu8a+YNPonhCD3icpiN4ddbsJoMZKcloTP6yOlfDJdB1xB534dSE5LPP+AfuR+cP9KzMVzzJehlf8gLjKVVtxON/+7dChbV+7A4wr+WTSaDAWaKRWFo5ydCfvHYTSp3IHiRHqWIo/cCTIjPhOKVLQq/4Q3NET1UfUXD8HYx79i6rszCzxRRVJzyGwx8chH99C6a4t4i1eqkHoGuOcQsxIQDoS9Z1xkKs18+uy3bF25PaS5B4hYCUAgrHnXxr3UalAjFvEURSGsRGUaDT1hzDOoEhNBOLTnCJNH/xR2zHUovG4f9S+oGyepSjH6fmJ/5hAgksHSPg4ClV48Lg8/fvBLoUogWoQm0H3FU+ZYkQfjOUC8WoIKMDWJeRalCILw09hfoy47fRzNIGh9lUrIAUDYiDxKKC9mEKmI8p8hRGJvYtcv2YwIUXcoVnweP1VqVy6WuRUnEMIAjjuAOOQlCSvC0T/maZQiCML6JZtjfuJKqZDMoHdi/wOVBaSoQthmIUMdMLUksN3VQJQDx12ISj8hjKrPs8fpKZZud5pB49JeF2JPtsV9bkVBhONOMDcnNmWggbkDmC+IWZ7EfrwKQTycZe8sHplwkUGhEHIfEo2w7KLSg1bh65zEMR+BiGTFcaqdVSWsSKBIEEJgT7Fx+7Ab4zqvIjRCGCFtLDLrU8gaC9EESworIvW1uDwYqB1BEC7s1hJrjOUkHm77DFPfnRkyqiOhkD4I94Yu/UipI4SmlEAQqp1VhTqNakZ0jdlmDlxz8v0ip4Jp1TqVeXvecCrXLPmM/URGCCNa0p2IyvMg6VEi9huIlICZKR6yqPDRgnhcHnpUuRNnRmwx7xa7hep1q/LGn8NwpCRuwTQp3ch9FxBeDoEAzGDrjnAMQBhrFbN0pY///lrDE51fDNlD+2SSyyfx/d6x+Lw+dm3cy9zJ/7B70z4cKXba3dA6rAqmiuJD+jaDbxMy61vw/hnmVQKsXdFS34horVDho0oR5HDsYDrZ6U5SKibjSLGz6OelPH/Da2F/2UJhspho0bExL0wt0KYhodCPPQPOiQRyCsPBABjB1gthqgfmCxAJ2pwmGPOnLWb4TW+ClIV+Ri12M8OnP0mT9ueVoHSKcJCeZcj0YeDbSODzHk7/geNoiArfI0yNIlpTKYIQzJu6iC9fmMCW/7bnJuE0v6IRtw3thcflZfSgsezeuBeD0YDf54+qd7HJauKTNW9RpVbibr2lfy/yYDeQ6USeT5DTVMV0LqLca2qXkEPWsSxmfT6H2V//xY51u3FmuDBbTWgGDb/XT82GNRg05k4atK5X6Dwet5f0QxlY7RaSUh0lJH1iI93zkEfuIfzCiycjEJUXI7TwelrnXqUUQUE+fuprfnhrRoF8ASHAbDUz5PNBtLuhDVtWbmfH2l0YTUamfzCLFXNWR7RTMJmN3DG8Dz0f6RazzKUZ6duMPNwP5NEwyk8HQwORhKjwA8IYmZ08ETiy/xhrFqzH7/VT67wzqXlu0Iruuexcv5uvhk/kz+/nI4TA5/NT5/wz6fPkDVx8fWtlLiompPQg918YY2axEZKHoDkia+qoFMFJLPp5Kc/3eK3QJ3yL3czHq9/M50TzeX2888DHzPrsD/x+PeyuZb0fv5b+L90ck8xlASl18MxHOieBaxqR7w40MF+AVv7z4hAvYVj59xqe6PISHpcH3Z8/msvqsNDxtksZNOZOpQyKAemcjkx/JsxWlIVgvgSt/NiILgmlCBI2aujz578v0syj+3SmvBMohupxefjlizkMavMkf01cQHL5ZGrUq4ZmLPq/0Gw1qVDSHITQEJa2iOQhRJddqYNnKdK/O96ilXkyjmSycekW1v6zgae6vowry1VACQC4stz88vkcZn/99ymQsuwj3b/HrgSAeJapSMg8AmeWi7ULNxQ5zuvxMfubv7jugS481O4Zjh3MwJV5wqZ39MAxdF/RfwwpA31jFXnQUoj6gyxMgYbdhjPiKlJZZfvaXXzy1Df889O/GM1GPC4vPk/hTntXlpsvX5xAhz4Xs2ruWia8MY0N/25B0zSaXnYe1z98NXXOV+a5qJDx6E5oBlPkbUtDkZCKYNnslWGPdWa6+N+lQzmw41CBp6dwzEJmm5mLr2utdgQnIYQFaWkffUVS1aksLNYt2sjgy5/HleVGShlRxvz+bQd4sstwVs5dizvbw3Ez8qzPD/L7+LlcM6gLd758szIfRYqpYU4RxtgUgrD3jo88JKhp6Jcv5oQ91mwzc3DX4aBb6LxoQeq/WJOsNGhTj0fG3hOxjImASLqf3IigSJBeMDWIuzxlDZ/Xx5NdX8KZ6SIaX6DH5WXZnFW5SuQ4ul/H7fQwZczPTHr7x3iKnBAIW09iq8RrA0d/hCF+daESUhHs2rAn7LFH9x4LK6VfaBoNLzwHi92CxW7m3Nb1GPLZQEbOegazVWXIBkOYGoKlY+QXGuqpnIIwmDt5Ed4Ya2b53KFNSO5sN188PwGfN9zcEAWAMFQC+61AYXWdBAGDTZ7MYWEDLJA0AJH0UFxlSkjTkNURfvmIcJ+kLHYzD70/gDqNVIx7uEjXL+D+JfILzQ3jL0wZZOYns3FmxtZfuyh0XWfxzOW0uTqxe25EikgeHNgTZH9JYHdwcuCKJKAE/GA8D0yNEab6YO0Wce5AOCTkjqDDTW2xOqIwSRSCz+PDnsBlJIpCSg/SOR398N3oh25GP/YUMmMkUSXUOCcj/YfiLmNZ4psRk1g8c3mxr+P3+TmwU/0tIkUIDS1lCKLSbzm7A1OQUW7AB77NgBth71MsSgASVBFcceulcXdwVa5Zkco1K8Z1zrKC9CxH7m8biJ32/A7eReCcAP7tUc4okM5v4ypjWWL6h7/w1YsTo/ILRIqmaap0dQwIQ+WcUNLC/lZOcM5AetcUmxwJqQgcKXaenfBoUAdvNFgdFm5+uoeKnsiDlB6kfhjduxZ5pC/IYyfFTsdyk3LHsfl32cLn9THuia9i7q4XLn6fn1ad4xfGmGhI6QLnFIquweUNlKwuJhLSRwBQp1FNNIMBXY/N0SU0Qae+7bn85nZxkqx0I70rkJnv5YTHaYCXmHsVByVhP7qFsmTW8rByW+KB2Wri4utbk1KheMwVCYF/JwgtjK+IH7zLik2MuOwIhBCdhRDrhBAbhRAFymwKIf4nhFgthFghhPhNCFErzzm/EGJZzs/UeMgTDhuXbsFijy2aRwjB5TdfwsDR/dVuANCzJyEP3QLu2QSecDwUjxKwg/mSYpi39LNny/6wo3iEJjCYoq9nX71eNR56f0DU1ysgEB0U7nek+Aw4Mc8sAp0R3gG6AA2Bm4QQJ4d1LAVaSikbAxOAV/Kcc0opm+b8dI9VnpLEbDVx76i+SgkA0rsO0ocScP4Ws21aSITt6uJdo5RitVvQDOF9ra8Z2IWLrmkVVeCE0WTgwffuwpak/AMxYTiT8G7DJjAXX3WCeKiYC4CNUsrNUkoPMB64Ju8AKeXvUuaWm1wA1IjDujFxdtPaMfUlttjMPPbZQFLKq20xgMwaR8AMVNyYIWUkQlMRWsFo0akJ/jBMQ1aHhcv7XMwz3/6PwZ/cT7WzqkS0jsVuwVtIjoEiPIQwg/0miq67pSEctxWbHPFQBNWBHXle78w5For+wE95XluFEIuFEAuEENeGukgIMSBn3OIDBw7EJjFQ8YzyNLm0IaIIh7EwCGxJVuwpNhzl7FhsZuo2r8PwH5/kkh6qflAu7plAeJVYCxIsdC4ExsZots5RrlP2qVSjAs0uP7/QvttCE1SpVYlzL6iHEIJLelzIwNH9sSWHn1/jcXtQG+HYkL5tSPefYGoFWmVCfg9ETiaxsXaxyVKiHjchxC1AS+DSPIdrSSl3CSHOAmYLIf6TUm46+Vop5YfAhxAoQx0PeQaO6c/9rR4nO90ZNNTOYrfQb/hNXH1PJ9b9sxG300PVOpWpUa9aPJYvW8hYEpcEYAfC6FEgj8awTmLw2KcDua/VEA7vOVqguJxm0HCUszNsypB8x89rWx+/N3wns8/j58mrXqJyrUrc+kwPLrvpYmUiDRPpWYxMfynQmUyYAAnSD1oV0PeBMAZqaQkjoEHSgwj7rcUqUzx2BLuAM/O8rpFzLB9CiCuAp4DuUp4ovyel3JXz72bgD6BZHGQKi+p1q/Ha789R/oxUNO3Ef4XFYcaeYuPeUX25/sGumC0mGrVrQMtOTZQSCIWI1kRmhaRBBRurh1xHlesoinIVU3j/31e56q4rsDoCu1l7ig2z1USHPhfzwbLXOOPsqvmucaTYaXRJ+PWbpB4oYLdz3W5G3f0BoweOK5G8hdKO7pyBPHw7+FYCrkBzGpkJOANKQKsE9nsQyYMD3fgqL0Bz3FbsSjYeO4JFQD0hRB0CCqA30CfvACFEM+ADoLOUcn+e42lAtpTSLYSoCLQlvyO5WFnyy3Ke7/E6SImuB56GhCbwe/y0692Gznd0KClRSj+2npD9OUX7CUwgrIHCccbaiOTHwHwhMutjit4RWMGqzELhkJyWxKDR/Rnwyi3sWLsbXdepXq8ajhDZ716PlzULii7NHoxA/4I/aNbhfNrd0CYWscss0rcRmf4aeGYXMsoL+m7IehtpaoFIeRIhIjCbxkDMikBK6RNCDARmEiiO8bGUcpUQYhiwWEo5FXgVSAK+z9Fs23MihBoAHwghdAK7kxFSytWxyhQOqxesZ+h1rxRoTiN1iU/389fEBQhN8NgnA0tCnFKPcNyGdI4P3OBDD4JyIxHCDoYz8hWOk46+kPkeRZXmFfZecZI4MbDYLNRtVqfIcXMn/RPoHhclriw3Xw2fqBRBEKRnMfJwf8Ivp+IH7z/IQzdB+bEIc6viFA+Ik49ASjkDmHHSsWfz/H5FiOvmAY3iIUOkvPfwp4V2KHNne5jz7TxufuoGqtcNbg7as3kfW/7bjmbQqN/qbNKqpBaXuKc9wlAN0sYhj9wJ0kf+D70VhBGRNg5hLmj5k1KCSKLwRjVWKPcyQisfZ8kVAItnLceZEVuBum2rdpBxJJPktKQ4SVX6kXo28sgAwBnF1U7kkXuh8rxAdFExkpDpmXs272Pzim1FjtP9OtPem8k9r9+e7/im5VsZff9YNvy7GaPZiBDgcflo2akJg965k0o1KhST5Kc3wtwcKv2KzP4Wsr8PlJUQ5cDeC2HvFfQmLqVEHnscXD8T0qwkqiJSRyAsFxXvG0hg4lFK2mAy4MpyK0WQF9ePMTZR8oPrF7B1jZtIwUhIRbBzwx5MZiMeZ+E9i31eP5uW5VcY6xZt5NEOz+PKCjw95c1FWDjjX1Y3H8y7i0fma3ifSAitPCLpXki6N7wLXD8GfgqU4c2DPAqGU556UqYprO9AuOh+SbmKKq8mL9I5mbCi4UJOkIV0/4IoZkWQkEXnTGZjoJFwGJhtJ7Zkuq4z9LpXc5XAyeh+nYwjWYzsOyYuciYCgVLUhStk8CGzPysJcRKSrat2MH/a4pjm0Awa7Xq0UU2YTiYeTeplNGalyEhIRVD/grr4wsm+TLLS7vrWua///fU/stML1+66X2ftwg3s2bIvZjnLOrpvdyBkrkh84JxV7PIkKhPfmIYvjP7bhWG2mrj5qRviJFEZwlCT8GOjg2EC4znxkiYkCakIbA4rHW+9BJOlcMuYANr3bpv7eskvy8Pq+KQZNJb/USLBT6Ub58Twx8qjgdLW7jlI5ySk63fypKMg/XuR7oVIz9J8xxVFM3fyoiJ7cheG1WHhxelPUPPcwgoKJCbCcWtOi8moZ4hrk/pQJKSPAOCuV25l+ZzV7N64F78v+NNQWtXUQGamPVCUy+MqyoQRQOqyQEanIgiefyMY7Ebua5nzu04gUlkgrVcFGtx4l+ckm0lAR9p6IpIeVjWJwsDjDu9zHQyj2cjrfzzPOS3OjqNEZQhTy8ATvXcVkdfisoLtaoSh+BVsQu4IAOzJNu5+/bZCsyEPbD/IE12G5445q1GtsCo1agaNmg3U01GRRLxjduX8eAiE42WDawJ4/wHcJ7I0ZTZkf4M83Aupx8FGW8YpXzUt6msNBo2aDZQjPxRCCETaWDDWp/Bm9UGwdkGkDCsWuU4mYRUBwFcvTCh0S+z1+Ni6agcr/14LwGU3XYzUi3YyO1LtNGoXfrp+wmJuQUQF5yLCA76tyIxXi2n+0smujXt4+/6PuCb1NjoZe3FtWt+Iis3lRTNoXNa7LVZ7fPt/lzWEloKoMAHKvURETz+OOxGiZIw2CasI9u84yKZlW4sc585yMXl0IFfOnmzj1ud6YSnkg2+xmXngnbtUAa4wELYbi3kFDzgnIfUYwvfKEAt/XMLdTQczY+xvgUKLuiTrWDablxedU3MyQoAt2cotz/YsBknLHkJoaLauYIjAhHboBvTMsUgVNVR8HNhxCKO5aG0rJezedCKypdej3enz5HWYLKZ8oaVWhwWL3cIj4+7jwm4tg02lOAlhqASOe4p7kYD/IMHZuWEPL9w4Cne2G380EUJ5nmtsyVYqVq/AW3+/SJVaiZkvczLSuwL9yAPo+5qg722Ivv+SwE1cTz8xRkoQkfisXJD5NvJQD6SeEX+h85CwzmJbkpXs9PA0rT3lhG1PCEGfJ2/gqruu4OePf2f1/HUYjAZadGzC5TdfrDo2RYhIGoj0LAHvvOJagaLqFyUCE16fhs8TfeMgTRO06doSR6qdy3q3pUWnJvkq9iYyeuaHkDmGgO8qx9Ss7w3cxLPHQfnxCGMtZNb74Fsb4ewu8G1DHnsMkfZenCU/QcIqguVzVoU9tvGlJ3fehNRK5eg9JGQfHUWYCCEg7U3kwa6gH6LwekNRIL05sdyJzeyv/wqrc1kodL/kyn6XcVH34i+AVpqQrt9ylECwsHIX6B7k4VuRFX6AzLeJrnmTB9x/If17EYaqRQ+PgoRR6VnHslg5dy3//bWG9EMZjHv8q7CvjbY8ryI8hJYacKYZ6uRsnY/bIeLw8TTWzlflNBGRUoaV/1IUX7/0QxykKVvIzDcpvKqoDnoGHLqO6Dv4AWjg/i2G6wunzO8Ijuw7ykePfcmc7+dhsgQiVNxOT0Rx/ktmLadH5X5cfe+VdLunExWqRR9upwiOMFSDijPAuwTpnBLYHRjOCBSvi6pyYw7JQ4oeU8YRQmB1WHBlxWYi27h0C1np2SF7GiQa0r8LfFvDGJkFMQcseAIKpZgo0zuCg7sPc3ezwcz+5m88Li9Zx7LJOpYdVbLXsYMZfP/qFO6o/0BEZiVF+AghEOaWaOVeQEt7Fy3l6djMOiIVzdK26HEJQIc+7TAYY/u6GwxaXHYWZQb9cE6ryXCItXubNaevcfFQphXBiFveJv1gesjM4UjxuLw4M1083W0E+7YdiMuciiKwXU0gizhSLJB0f7ylKZX4/X4MRi0mHwEE6millFclpnMRqYU3YoorfrB2KrbZy6wi2LN5H2sWrI/5wx8Mn9vHxDenx31eRUGEvSeRpyCbwVBDdTMjUDH3xRtHMeuzOTHP1fiShqq6aB6E8cwSKo9uBXsfhFZ8SrjMKoLFs5YXW1KXz+tj5ie/F8vcivwIrTykPBfZRaZmiArjETEV+yobzJ28iMUzl+HOjj2E9o7hN8VBorKFSH6Q4nO1ikDBOsvlgd7exUiZdRZ7nJ6YKioWRXa6EymlyiAuATR7L/TsH8AXTpE6M1guQ2jl8h2V0gP6QcAEWsWE+bt9+8rkmJ3EAC06NubcC+rFQaIyhqUDsZWZzsvxeSyB6DlzE4SjP5haFfvntcwqgjPqVsVkMeEtpiqgFps5YW4mpwWOPnBsGUXnGXjAuzT3lfTvQWa+B84pge+Z9IOhEtI+INA+U0TjfygdSClZv3hTTHMIIUitnMLjXz4QJ6nKGO65ICxh+AoEAV9XqPuRAaw9EMkPIQwl3+q2zJqGLujSDC3GKIlQHO/GpCg5hLASdvXGnJu77lmFPNAZnN8BzpxOTx7w74KMEcgjA5Cy7JYLl1KGHawiDILUyuWwJdsw28xYHRZMFhMtOjbmg2WvkVqpXNGTJCL6gcDDRZFIMNQq5LwfXFPAty5ekkVEXHYEQojOwFsEVN5YKeWIk85bgM+BFsAh4EYp5dacc08A/QlkWzwgpZwZD5kMRgMDXr2Vdx74GHd26Hrr0URTmCxGeg2+JlYRFZFgakx49dztYGqDnv4qZH9UyDgneBYhM99FJJfNp11N06hUowL7dxwscuyZ51TnwxWvsWz2SnZv3IvJaqb5FY2ofGbFEpC0FKOlgNDCULhajsIwEnpX4EIeHQSV5yNEyTrlY35kFoG99TtAF6AhcJMQ4uSaDP2BI1LKusAoYGTOtQ2B3sB5QGfgXRHHvXqXfpdz+wu9MVtNmK0n4n1NZiNmq4nrH+zKHS/eRKUzK2A0G7En2+hwczvuHHFzwPSj5Tf9CE1gsVt46IO7qXO+KltQkghDVTC3puiPrB7IwMz+NIxZXZD9GbLEQgBLnh6PXI3FXvhNxeqw0GtwdwyGQM2sbvdeSec7LlNKIBzMF4e5IzCBvovQSuA4OtL5M9K/C+nbWiKVRwFEYY1ZwppAiAuB56SUV+a8fgJASvlynjEzc8bMF4EC23uBSsDjecfmHVfYmi1btpSLF4ffbPvw3iP8+OGv/PvrCnRdcn7bc7nm/iupXDN05cSNy7YwfsRk5k5aCAQU/kXdW3LTE9dTr3lilyw4VUj/LuTBawMNaIL6Cqxg6wmuiYHmNOEgkhBpHyDMZbOGjjPLxT3NBrN/24GgfYlNZiM16p/BmIUvq9DQKNGPPZ/TdjVUsp0RtOogDwUaJxWJATDl7DR0sHVFJA1CGM6IWVYhxBIpZYHyyPFQBD2AzlLKO3Ne3wq0llIOzDNmZc6YnTmvNwGtgeeABVLKL3OOjwN+klJOCLLOAGAAQM2aNVts2xZ5DfVo8Pv9uDJdWJOsGAxl17FYWpC+Hchjg3Na/2mADHxhRDIiZRgyc1SEFR4diNQ3ENbLikniU8/RA8d4pvtItvy3Da/Li65LNIOGyWykfqu6PDdpMMlpKlEsWqT0IA/3A+9/FCyHYgGtAiTdBxkjwlQEJ2MAYUeUH48wxRa5FUoRlJqoISnlh8CHENgRlNS6BoMBRzlHSS2nKAJhPBNRYTzStw08CwEfGOuCqVXg96OR9jfIQrrngKkRwlA2TSGplcoxev5LbFy6hV+//JNDuw9TsUYFOvVtr0yccUAIM5T/BJyTkVkfgn8nyZ10tQAAHXJJREFUgRyAZHD0RdhvBf+eGAIT/CAzkUf6QaU/iiXSLR6KYBdwZp7XNXKOBRuzM8c0VI6A0zicaxWKAghjLTDmj8KQUicnRjSyyZwTkM6JyHIjAl2kyih1m9WhbrM6p1qMMokQJrD3RNh75nTE00E4ToSYa8lIY03wrY9yBRnYTXj+Akv7OEl9gnjEVy4C6gkh6oiAq7s3MPWkMVOBvjm/9wBmy4BNairQWwhhEULUAeoB/8RBJkUCIoQFtPJRXOkB3HDsCaR7QbzFUiQYQrMjtKQCeUYiZSgQXX9oAGQW0jktNuFCELMikIH9zkBgJrAG+E5KuUoIMUwI0T1n2DigghBiI/A/TjiJVwHfAauBn4H7pQzLBa9QBMd+OxBtM3UXMmNkHIVRKE4gzK0QaaOJyRAji6cUdVx8BFLKGcCMk449m+d3FxC0y7WUcjgwPB5yKBTC3geZ/U1OqF4U+DYhfdsRRmU7V8SXQM/iZDA0BP9/RF6a2gCG2sUgWSlyFisUhSF928G3BhCQNgaOPgD+HZFPJEzg3w1KEShCIKUE7wrwzENKD8J4Nlg7BkyToa7x70YevjPwgBJ1boAJYb8xymsLRykCRalGetci04eCd/WJJiHSA6aWQDL41xFZi0A9UPFRoQiC9G5AHh0I/r2AG9CRwgHpzyCTBqM5+hS8Rj+CPNQzxp7cFrBcGFA6xYBSBIpSi/SuRB6+5UTymMxTZdO7AHCAtRu4fibwBQxdauQERjCdF39hFaUe6duMPHwjyCzymXVkVuDfjJHouNAc/fJfl/Ux6MeITgnklKI2Nkakvhmt6EWiFIGiVCKlRB4ZWEgGsQ5kgW8zovI8pGsWpA8l8BQXCmsg7luor4WiIDJ9WEElkA8nZIxC2q5DaIG+5lL6IftrwnsIMYH5EtDsgeQ06QVjPYSjH5hbF2u1Y/WJV5ROPP+APFrEID1QzVHfh2a/HmmsHUjKkU4KfpmtYG6GcNxdTAIrSjPSvxc8SyjawetDZn2NSA60SZXOGeFH+ggHWvn3YpIzWspsGWpF2UZ65odZT0jkZCCDMDdHVJgAlo6AGUQSYAOtEjgeAPttyMN90feej763IfqBLsjsiYGmNorExrsmzEb1fsgeh5RO9OwpkP5k+Gucwt4YakegKJ3IcLtu6ZAntV8Y6yLSxiD1dPDvAWFBatXh2GDIHpNfufg3ITOGQfZnUP4rhJYc3/egKD1EYpaRWf9v78yjJKvqPP/5voiMNSsrK2tjE5EGZHFGxEKhFKRp+ugwM4AiWCBC0VBsRxt1cMR2WgePju20Cu3CyNoUiFCKOBSLrWwFiA0tIkuhzdrDKFNUQVXWkpFbZMRv/rgvs7IyIzJexpqReT/nxIkX8e5773dfvHi/d+9vw3Z8C/rXMPVU5HgCSBxZjWR1wY8IPG2JOg505fwqN4QSnhYKulDH21F8H8h9D4YeLD3CsAEYecXliffMXeIHO2+0SBj03zLNAyScLaBFeEXgaU9SH4zWTukpn7TMXE2CyVkjxzMMw7/FRmor++hpXxRbMs0n9jzRRwNJyJyBOt5RhWT1wSsCT1sipaDzc0yduyUF8y5DmuIyH3qUaMXHC9jAHdMT0jOrcLmCGnDLzJyO5n2u/vudBt5G4GlbguzHKZKHHd/C/UFHn+ozgEHXVwnSfxlGgv4O678RRl4BpSB1PEp/BIpbIlaYGoHipob1xTPzUXwvLPkXMHRvHffagTrPb6hraBS8IvC0NUF2JZb+CDZwOww/6YrUdByJ0v8ZBRms2I/1XgAjT4MNMub+l38e23G5q2hWtrLULkcCzc56BR6wwkYYvBsrbELBAizxfpdaWimI7TU2qlTnRdjQI0S7ZiohSLwfVZUxt754ReBpexR0oexKyK7c5Xszw7ZeCPknmRzQE44eBm6KeJQiDP+zyy3T5MLinsZhNoRt+wIM/jL8ZhhDwLew0ZKRQReWPRdlPoE6DsGSR7u62NNKXVKKJOpq7ZTQKN5G4Jm95J+B4aeYOqpzGhkgR150boGeWYFZwSWCG7wPd42MXiej10QBGHRTgju+jfWeh9kI6v42xOpQ4Cf1QRTfr/b91AGvCDyzFuu/keieG1EYhIFbsaqzR3pmFEMPwMizRJvmGYTh32C5a5ESaP5XqanIDDh71QzBKwLPrMPyL1DcdhkMPkD12R7LEQNfxWxWYLlrIkanjzIIuX90+YM63gWx3YjmcTbz8YrAM2swG6TYez62+aMwcCuQa8RRXO1YT/tTVf3gYRh5HklowQ9coZmqbqMdkDi8iu0agzcWe2YFLhvpRTD8G+o7HVSCYElj9+9pKGYFV3xougXCAAiwYg6Gn8AG7oKOw6Dwf6HwJ1AS96AwQGVDcoAyZ1QjQEPwisAzO8j/LswO2WAlQAoSyxp8DE8jMBvEctdC7iacXaCKa8X6YNulmG0el8U2DgQQ3x/mfRaCpbD5Y2F23FJTk2nIfALF96qhN/XFTw15ZgWWW019fLunIg2df41amCXSUx1WzLkqYX1XgfWGN/Fq7EcGxT+GtoXRIcUIMAz552DHFSi2F1p0u8tPRIqx521lXMqTzovQvP9Sh17VDz8i8MwORl6kyrF+ROKQ+gjKrGjgMTyNwrb/dxj5N6IViKmWIRj5PQw/gpLHoEW3Y/kXYOghty62l3MZnYGlUGtSBJJ6gDXAPsD/AU41s94JbQ4F/hfQhZs4+5qZrQnX3QB8ANgWNl9pZk/VIpNnjtLwIC/B4E8w2wjz/x4F2QYfz1MvrLg1LFfahLoS1o/lrkPJYwBQxwHQcUDjj1sjtU4NXQrcb2b7A/eHnyfSD5xpZocAHwKukNQ9bv3nzOzQ8OWVgKc6UscByQYeIA8Mw9DD2JYzfLGadmLoV9DM8qP5F5t3rDpR69k5ETgmXF4NrAM+P76Bmb0wbvn/SdoELAYq1Rn0eCKj9Aqs76oKrVIQ7AbFV6l+GmkYRp7DNr4Xy5wFiXeiYi9GEQqbYfC2sOBNHBLLUXYVSryrymN56oLlwOodTzIFbWhDqlURLDWzDeHy68DSqRpLeg+QAMYndv+apC8RjijMSpeeknQecB7A3nvvXaPYntmGYouwrq/C9r+ltNE4CR0HQefF0HshU9cfiEIO+q+E/jIqxYZh6H5s6FEscyZB18wyDs4pYnu4ZISRdL+ozdYUQGJ5Ddu3hopTQ5Luk7S+xOvE8e3MzJjiDEraHbgJONtsTD1/ATgQOBzoYcJoYsL+rzazZWa2bPHixZV75plzBJkT0YIrnRsfKRfso05Q1hX+6LmJILkcMiuAZhjsDBiA/tUU+3/WhON5SpI4EohSb3iUWmbMEyh7dg3bt4aKIwIzO67cOkkbJe1uZhvCG33JhO2SuoC7gS+a2Vh8/rjRxJCkfwQumZb0Hs8ElHw/St6NjbwEhdecy17HO3fJGKp5l2Lx/aDvO2A7gBhQcAqj2ItzB6wng9D3D1j6pJbnnZ+LSHGs89Ow42s4W89UBC5gsLihQrtSpCFzJuo4uIptW0utxuK1wFnh8lnApBJOcv/AnwE3mtltE9btHr4LOAlYX6M8Hg8QFqlPfgAlDp+UNloSQeYUtPhh1PND1H056rkJLX4EOv5dYwSyrTDyh8bs21ORIHt6mA6iEgX3MND1TcqPImK4+hThaFNZUA/Mu7RtpwBrtRH8HfBjSecArwKnAkhaBlxgZueG3x0NLJS0Mtxu1E30ZkmLcRNzTwEX1CiPxxMZSTCxTmx2Fbb1EpyzWz2JuWpoFTArQuHfwIYgtgcKuitu44lKpdHAKEWUOgo61mA7/j4seJTABaDFIXMmZM9Ew0+6EWWwBBKHt3WgodzUfnuxbNkye+KJJ1othmcWYlbEes+H4cepa6SyMm70UaZAuVkey90A/deHUa+BMzgnj0Kdn3H+6CW2ofiGaxssbusbUT2x4g5s4Gc76wzED0LZM7AtZ4bnKwKaD/EDUPZcLH4gKr7ucgnFD0DNdEWtM5J+a2aTcqR4ReDxTMAsj23/Ggz8FPcUWQfXQy1ESx4dK3kIODvGyEuYCfpvcCkKJikfASnUcx0KcxxZsdelUO6/Nay3bC51QfYslD17RkauNoti/09h+5fDT6OxHnEg5ryHCn9kejagDHS8A/Vci1Rj/YEZgFcEHs80seJ2LHc15K6ndgNyHBJHoHn/1UWfbr/MpTxQ3E0DVZq2UCda8msobnNptou9TI6UTUJ8H9RzCwo6a5S39djIS1juehj8has3HSyAzOkos2JSnV8rbMK2Xgz5306xxyTVKfYkJI8hWPDdaW438/CKwOOpkmLuFtjxddyNt9bRwajhepqRycqgri9huZtdPpuyaY4TkDqOoPuKGmRsPe7J/jLcjXt8X5OgpDPudxwEgBVexzZ/GIqbGyhRAi36pxmVMbQayikCn33U46lAkD0NLbwFUh9i5428WsbXxp0G1o/lbgmT602V634YBu/HIhimZyo2/GSoBAaZ3NchsO3YljOxoisQZNv+JpIhvkapsMFJTpGzBq8IPJ4IqOMQgu4rCHZbDz3/hHMhbDLF14mkRBSDoXWNlqZhWN/3qGiotz6s72qssAGGH6OxmWcB8q6YzSylfc3fHk+rGLqbsSC0phE4f3U2Vm5qBSg2okxn47FiX3hjr0QB+q/ChtZR/wDAUgTORjFL8SMCj2e65J+mKSmNx6MkJD9ApAyrioeF1dsQ2070dBAGhecbKc04Eih1fJOO1Xy8IvB4ImL55yj2fgqGH2nykRPQcSjKriLaFIhCpdFeWGEjln+Z6SnZZji7xCG+f1umjoiKnxryeCJQ7F8L2/8b7ibVLE87ufiA+CGo+0oUZLH0STCwlvJz6GnIXjgprcZMxvLrsR3fgOGnwgjemebJGED391otREPxisDjqYDlXwiVQKNrIo8iZw9IHImyK6Fj2ViyOnV92VXcGn7E+daP3TRjQAdkPoay5+6UvbgF6/8JDPzMFV4PlqLsJyD1H5DqU8jHbACGn8aVY9wHxd8afduhX2O9FzB2bktnoW8xAQw9APGPt1qQhuHjCDyeChS3fh4G19Jc47BwBuK0UwSdq8YUgplB/kksdx3k17u2ife6dMv5p6H4pouije8HO/5HWJRlnBJTBtSFem5G8bdULaEVc9iOb8Lg7ThFJJcWI74/6voiSrx76u1tANu03BWOmekEe6DFD7Z99thycQR+RODxVGLw51SnBIT7i4npG5fNHdP6YPghbMvjkD4Bur7ibkaJd4/daK2wEes9F4Z+ET5RF3HmvzLBb9YPNohtOR0W/3JSSgqzYRh61EUvBwsguXzS6MGKOWzLqTDy6uS+jazHtpwNC74zVrt31/0bkIeBO6FdHkSLb7pYhdjCVkvSELwi8HimwN20qpgSUgaChVDsB+utVQpgAAbWYrG93ehgdE1xO7b5lDCZ2nhlVSkCuugyZw7cA5mT3b6siPV91+U9QuxUKIZlVqLOT43lSrK+b5dWAmMMYls/DYsfRUHWbTP0OJa7Cob/edy+6zHKEm5EUo0baYJoSjogevbS9sN7DXk8UyApYh57gBjE3w6JD6D5l0PHe8G2UZekdQAMQO4ql3U0xPpvDvMOVXFDtX6sf7VbNMO2ftblVbKcG4lYf/ieg9z12NbPuHY2GCbkq3ADtTw2cBcAxR3fwXpXwfCvQlltGjIHlHcpTUHX1yExabYjGspGbBfAhPxGswmvCDyeSqRPpvLgOQHZswkW3UnQc42bsx+8i/oHOxV2DbjK3QjUYGAthEUFh9bB8DrK13IegOGHnNE0/yzRbh15yF2HDd4L/ddRvbE9DumPhoVgOkPFnHLeVAuuIch8BHVfyfSjvROQXO5GbxWPf3JbeWJNFz815PFUQNmzsIEfg01xU1ccZc7c+bnwUphZtN7SmJuvBvdkXuu0U2gfsNzVbgQw5aH7sdw1qPOvcdMxESi+iu24PKyxUCUdBxPMvwzr+qJTQjYIsb128U5S0InF3ubO+3TovAQKr4dG91IKVS7za/b86uVvA/yIwOOpgGJ7ogXXhE+OE10uk6AsWnAdGh/N2zAjqLDhZym++Z+wjUdT27RTwhmgIYyWjkD+GSy2pLLSGMOg8HJV0o3R6co/SgmUeDdKvq+ki6o6LwCi1mJIQ/pkgvieaMF1kHgPkGKXUYWyzlto4RoUW1JbH2Y43n3U44mIFd7A+m+FgZ84Q6vmQfoUlDkNxRbt2ra4w7lG1jJtU5Y05adwpkMKLf4laB626TCiD1+SNDWwLrYfWnRnxQpszs5xMQw9RPnzE9ob0h9GXV/eZZ828hLWv8YVrwnmo9QJLpZDs+d52dcj8HiaTHHrJTB4N82NP4iOuq+ExGHY5o9B4dVWizM1wRI0/5soecSUzcyKLr4idy0wDCacbSIO8b0heTRKn972dQWqxccReDxNRvMuwYYeDhOp1eo5FKe+huc4Sh1HcfNp7ZFeubgJ6z0Puq9AqWPLNpMC1LkKy/6Vm/e3PggWuyC3Ng8GaySzZ8zj8cwwFNsNLbwN4vvjpnNG/27TfP5SlvqPKha41Bn552gf//hBbNtnsWJl+4QUQ4l3OntCxwFeCVSgJkUgqUfSvZJeDN9LJuyWVJD0VPhaO+77t0l6XNJLktZoNvtneeYkiu9NsOhOtPBHztsmex5kziFaquUUdByxs0B9XXkT27KKpqfTrhUjdMv11JNaRwSXAveb2f7A/eHnUgyY2aHh64Rx338DuNzM9gN6gXNqlMfjmZGo4xDUeRHBvEtQ/G1EGxUMQv43NCbZnYFtINqUVX2S0zniOO+cBASLKjUuQT829EAd5fFA7YrgRGB1uLwaOCnqhnJjtWOB26rZ3uNpW9ThIlWjNZ7uzqcrTYRdxt08e12IoyWPEey2nmDJr6HzYqYt81TxHJ6qqFURLDWzDeHy68DSMu1Skp6Q9Jik0Zv9QmCr2div+idgz3IHknReuI8n3njjjRrF9nhaSOJwsCjz8gmm9xeN0TCXzsyFRPfRn4L0CSjYGcmr7HkQfwfR7SYJ6Dikdjk8u1DxKpN0n6T1JV4njm9nzg+13FX41tBl6XTgCkl/Nl1BzexqM1tmZssWL67X04nH03wU2y0MYKqQEkFJN3qITL1yGo0nDZmzUfYU535JLWY8QbBrYJbUgXpWQ+IIoo4MlDmtBhk8paioCMzsODN7R4nXHcBGSbsDhO+byuzjtfD9FWAd8C5gM9AtafRRYC/gtZp75PG0AZr/dQi6Ka8M0tD9fab3hF/LaKDETVhpSH0QdX4SKYl6fhQqsOkopwny5a7CCrv+zRV0EvRcD/MvZ+qRQRoyp+0awe2pC7VODa0FzgqXzwLumNhA0gKFycwlLQLeB/w+HEE8CHx0qu09ntmIYkvRwjsgeSyQ2JlQjaSrT7zwRwTJIyD9CaaekolRFy/w+P7Q8e/DojWdkDgKdf8Azf/GWGStgnnuhr3wLijtIBiBPLZ5BVbsm7QmSB+Pem4AzZ+QFTQJJCCzAs37QpXH9UxFTZHFkhYCPwb2Bl4FTjWzLZKWAReY2bmSlgNXsTMB+RVmdl24/b7ArUAP8DvgDLPKtep8ZLFnNmGFzZB/BihA/AAU33vnOiu4tAnDvyqR32dUQdSebkLdV6HUn0eXeeQVbPOpLmBr2lNSaZj3GYLsytL7tjwM3YcN3u8K7XQchNKnoJifEq4Vn2LC42lTzIow9ACWuyZMAW0QewvKrsL6b4CRF2s8QgItfXrKXD6W/1ds8B5X+yC2B0qfAFbEdvxdmNunwLSC3oI9CJasq1Fuz3TxKSY8njZFCiB1HEodN2mdDf4SqEURCDo/WVYJWOFNbOuFkH8eF3xWBDqwvu9D8s9R97dceuqhh2Hb3xI5yV7Re/7NJHyKCY+njVH249GrbJWkC2U+XnKNFftcXeL8c7igttEpoDwwDEPrsC3nguYTpE+Cnh9OQ3D/DDqT8IrA42lnEkeFLpnTrc6VAs1HC3+IgtKlOK3/Rii8Qflkd0MusdvQQwAEiXe6lBgVkZPbM2PwisDjaWOkGOq5CWJ7jFUbq7xRD3R+Ci2+F3W8vWQTM4PcaipP9fRjuWt37jpScZgkyp4bTVZPU/CKwONpcxRbghbdg7oug9h+lA7MigEpmPc3BEsfI+hchYLu8ju1Pld8JwojL+yUJbkcMiumUEpp6FyFEodG27enKfiJOo9nFiAlIX0SSp+EFfuxgTug/wZXj1cJSB6Lsn9VdgQwmYDoAWq7Kp6g6wsU4wdC7rtQ2BzWbh6B2BLU+RmUPn4aPfM0A68IPJ5ZhoIMyp4G2RpSMSjjppsKf6zU0OVOmkCQ+TCWPsnVKy72QtADsX19XYAZip8a8ng8k5AEmVUR7A4plC2dPV4Siu+HEoej+J95JTCD8YrA4/GURJmTIX4w5esRpF020cS7mymWpwF4ReDxeEriMoPeAOkTgWQYr5By78pC5/mo6ystltJTD7yNwOPxlEVKovlfxeZ9HoYedp5EwWJIHoWvLDt78IrA4/FURME8SP/HVovhaRB+asjj8XjmOF4ReDwezxzHKwKPx+OZ47RlPQJJb+AK4TSCRcCbDdp3s2j3PrS7/OD7MBNod/mh/n14q5lNqvDTloqgkUh6olThhnai3fvQ7vKD78NMoN3lh+b1wU8NeTwezxzHKwKPx+OZ43hFMJmrWy1AHWj3PrS7/OD7MBNod/mhSX3wNgKPx+OZ4/gRgcfj8cxxvCLweDyeOc6cVwSSTpH0nKSipLJuWpI+JOl5SS9JurSZMlZCUo+keyW9GL4vKNOuIOmp8LW22XKWkGfKcyopKWlNuP5xSfs0X8qpidCHlZLeGHfeZ1SxXknXS9okaX2Z9ZL0nbB/z0g6rNkyViJCH46RtG3cb/ClZss4FZLeIulBSb8P70UXl2jT2N/BzOb0CzgIeDuwDlhWpk0MeBnYF0gATwMHt1r2cfL9T+DScPlS4Btl2vW1WtbpnFPgIuAH4fIKYE2r5a6iDyuB77Va1in6cDRwGLC+zPrjgZ/j6lEeATzeapmr6MMxwF2tlnMK+XcHDguX5wEvlLiOGvo7zPkRgZn9wcyer9DsPcBLZvaKmQ0DtwInNl66yJwIrA6XVwMntVCWqEQ5p+P7dRvwF5pZZa5m+nVRETN7GNgyRZMTgRvN8RjQLWn35kgXjQh9mNGY2QYzezJc3gH8AdhzQrOG/g5zXhFEZE9gfPHWPzH5h2olS81sQ7j8OrC0TLuUpCckPSap1coiyjkda2NmI8A2YGFTpItG1Ovi5HA4f5uktzRHtLox06/9qBwp6WlJP5d0SKuFKUc4/fku4PEJqxr6O8yJegSS7gN2K7Hqi2Z2R7PlqYap+jD+g5mZpHI+wW81s9ck7Qs8IOlZM3u53rJ6duFO4BYzG5J0Pm6Ec2yLZZprPIm79vskHQ/8b2D/Fss0CUmdwE+BT5vZ9mYee04oAjM7rsZdvAaMf5LbK/yuaUzVB0kbJe1uZhvC4eKmMvt4LXx/RdI63JNHqxRBlHM62uZPkuLAfGBzc8SLRMU+mNl4ea/F2XPaiZZf+7Uy/qZqZvdIulLSIjObMQnpJHXglMDNZnZ7iSYN/R381FA0fgPsL+ltcvX5VgAt97oZx1rgrHD5LGDSKEfSAknJcHkR8D7g902TcDJRzun4fn0UeMBCy9kMoWIfJszjnoCb/20n1gJnhl4rRwDbxk1DtgWSdhu1LUl6D+6+N2MeKELZrgP+YGbfLtOssb9Dqy3mrX4BH8bNtw0BG4FfhN/vAdwzrt3xOGv+y7gppZbLPk62hcD9wIvAfUBP+P0y4NpweTnwLM6z5VngnBkg96RzCnwFOCFcTgE/AV4C/gXYt9UyV9GHrwPPhef9QeDAVss8Qf5bgA1APvwfnANcAFwQrhfw/bB/z1LGs26G9+GT436Dx4DlrZZ5gvzvBwx4BngqfB3fzN/Bp5jweDyeOY6fGvJ4PJ45jlcEHo/HM8fxisDj8XjmOF4ReDwezxzHKwKPx+OZ43hF4PF4PHMcrwg8Ho9njvP/AfDVWai4xWKgAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = Sequential()\n",
        "\n",
        "model.add(Dense(10,activation='sigmoid',input_dim=2))\n",
        "model.add(Dense(10,activation='sigmoid'))\n",
        "model.add(Dense(10,activation='sigmoid'))\n",
        "model.add(Dense(1, activation='sigmoid'))"
      ],
      "metadata": {
        "id": "tzfO3jquTVyY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])"
      ],
      "metadata": {
        "id": "bzYYyQhhUAbJ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.get_weights()[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CI5zBIvv3x5e",
        "outputId": "915901ed-1520-4748-ebbd-35f613b6791a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-0.33058235,  0.292853  , -0.5027271 , -0.01332176,  0.4663965 ,\n",
              "        -0.08549386, -0.3767269 ,  0.02521479,  0.17957473,  0.09508276],\n",
              "       [-0.46751493, -0.5052102 ,  0.618305  , -0.41437078,  0.51485544,\n",
              "         0.45250946,  0.36829787,  0.66284555,  0.14777309,  0.04143274]],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 91
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "old_weights = model.get_weights()[0]"
      ],
      "metadata": {
        "id": "_0MskkInUhLT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)"
      ],
      "metadata": {
        "id": "TaYrBqJvVePG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(X_train, y_train, epochs = 100)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ouf7UIKiV7hr",
        "outputId": "225985d9-84f0-4baa-b5b3-90a97d757327"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/100\n",
            "7/7 [==============================] - 1s 3ms/step - loss: 0.6966 - accuracy: 0.3700\n",
            "Epoch 2/100\n",
            "7/7 [==============================] - 0s 2ms/step - loss: 0.6958 - accuracy: 0.1650\n",
            "Epoch 3/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6954 - accuracy: 0.3050\n",
            "Epoch 4/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6950 - accuracy: 0.2800\n",
            "Epoch 5/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6946 - accuracy: 0.1250\n",
            "Epoch 6/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6940 - accuracy: 0.5100\n",
            "Epoch 7/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6936 - accuracy: 0.5100\n",
            "Epoch 8/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6932 - accuracy: 0.5100\n",
            "Epoch 9/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6931 - accuracy: 0.5100\n",
            "Epoch 10/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6926 - accuracy: 0.5100\n",
            "Epoch 11/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6922 - accuracy: 0.5100\n",
            "Epoch 12/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6919 - accuracy: 0.5100\n",
            "Epoch 13/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6917 - accuracy: 0.5100\n",
            "Epoch 14/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6910 - accuracy: 0.5100\n",
            "Epoch 15/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6906 - accuracy: 0.5100\n",
            "Epoch 16/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6902 - accuracy: 0.5100\n",
            "Epoch 17/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6895 - accuracy: 0.5100\n",
            "Epoch 18/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6895 - accuracy: 0.5100\n",
            "Epoch 19/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6894 - accuracy: 0.5100\n",
            "Epoch 20/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6891 - accuracy: 0.5100\n",
            "Epoch 21/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.6885 - accuracy: 0.5100\n",
            "Epoch 22/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.6880 - accuracy: 0.5100\n",
            "Epoch 23/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6873 - accuracy: 0.5100\n",
            "Epoch 24/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6865 - accuracy: 0.5100\n",
            "Epoch 25/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6861 - accuracy: 0.5100\n",
            "Epoch 26/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6856 - accuracy: 0.5100\n",
            "Epoch 27/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.6846 - accuracy: 0.5100\n",
            "Epoch 28/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6836 - accuracy: 0.5100\n",
            "Epoch 29/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6830 - accuracy: 0.6400\n",
            "Epoch 30/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6819 - accuracy: 0.7650\n",
            "Epoch 31/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.6810 - accuracy: 0.7000\n",
            "Epoch 32/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6800 - accuracy: 0.6800\n",
            "Epoch 33/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6790 - accuracy: 0.6750\n",
            "Epoch 34/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6780 - accuracy: 0.7450\n",
            "Epoch 35/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6766 - accuracy: 0.7850\n",
            "Epoch 36/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6754 - accuracy: 0.7750\n",
            "Epoch 37/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6740 - accuracy: 0.7750\n",
            "Epoch 38/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6725 - accuracy: 0.7900\n",
            "Epoch 39/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6709 - accuracy: 0.8050\n",
            "Epoch 40/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6695 - accuracy: 0.8000\n",
            "Epoch 41/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6675 - accuracy: 0.8100\n",
            "Epoch 42/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6653 - accuracy: 0.8100\n",
            "Epoch 43/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6632 - accuracy: 0.8000\n",
            "Epoch 44/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6608 - accuracy: 0.7950\n",
            "Epoch 45/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6583 - accuracy: 0.7900\n",
            "Epoch 46/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6557 - accuracy: 0.7900\n",
            "Epoch 47/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6528 - accuracy: 0.7900\n",
            "Epoch 48/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6499 - accuracy: 0.8000\n",
            "Epoch 49/100\n",
            "7/7 [==============================] - 0s 2ms/step - loss: 0.6466 - accuracy: 0.8000\n",
            "Epoch 50/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6433 - accuracy: 0.8050\n",
            "Epoch 51/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6395 - accuracy: 0.8000\n",
            "Epoch 52/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6358 - accuracy: 0.8000\n",
            "Epoch 53/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6317 - accuracy: 0.8050\n",
            "Epoch 54/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6274 - accuracy: 0.8000\n",
            "Epoch 55/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6230 - accuracy: 0.8000\n",
            "Epoch 56/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6186 - accuracy: 0.8050\n",
            "Epoch 57/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6135 - accuracy: 0.8050\n",
            "Epoch 58/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.6083 - accuracy: 0.8000\n",
            "Epoch 59/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.6038 - accuracy: 0.8100\n",
            "Epoch 60/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5985 - accuracy: 0.8100\n",
            "Epoch 61/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5927 - accuracy: 0.8100\n",
            "Epoch 62/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5869 - accuracy: 0.8100\n",
            "Epoch 63/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5810 - accuracy: 0.8100\n",
            "Epoch 64/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5748 - accuracy: 0.8000\n",
            "Epoch 65/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.5690 - accuracy: 0.8050\n",
            "Epoch 66/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.5628 - accuracy: 0.8000\n",
            "Epoch 67/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.5563 - accuracy: 0.8100\n",
            "Epoch 68/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5497 - accuracy: 0.8050\n",
            "Epoch 69/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5434 - accuracy: 0.8100\n",
            "Epoch 70/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5369 - accuracy: 0.8100\n",
            "Epoch 71/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5308 - accuracy: 0.8100\n",
            "Epoch 72/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.5243 - accuracy: 0.8100\n",
            "Epoch 73/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5179 - accuracy: 0.8100\n",
            "Epoch 74/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5119 - accuracy: 0.8100\n",
            "Epoch 75/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.5053 - accuracy: 0.8100\n",
            "Epoch 76/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4993 - accuracy: 0.8100\n",
            "Epoch 77/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4930 - accuracy: 0.8100\n",
            "Epoch 78/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4874 - accuracy: 0.8100\n",
            "Epoch 79/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4817 - accuracy: 0.8100\n",
            "Epoch 80/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4764 - accuracy: 0.8100\n",
            "Epoch 81/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4711 - accuracy: 0.8100\n",
            "Epoch 82/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4656 - accuracy: 0.8100\n",
            "Epoch 83/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4608 - accuracy: 0.8100\n",
            "Epoch 84/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4557 - accuracy: 0.8100\n",
            "Epoch 85/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4507 - accuracy: 0.8100\n",
            "Epoch 86/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4468 - accuracy: 0.8100\n",
            "Epoch 87/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4423 - accuracy: 0.8100\n",
            "Epoch 88/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4381 - accuracy: 0.8100\n",
            "Epoch 89/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4340 - accuracy: 0.8100\n",
            "Epoch 90/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4299 - accuracy: 0.8150\n",
            "Epoch 91/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4261 - accuracy: 0.8150\n",
            "Epoch 92/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4219 - accuracy: 0.8150\n",
            "Epoch 93/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4182 - accuracy: 0.8100\n",
            "Epoch 94/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4145 - accuracy: 0.8150\n",
            "Epoch 95/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4109 - accuracy: 0.8150\n",
            "Epoch 96/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4077 - accuracy: 0.8150\n",
            "Epoch 97/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4044 - accuracy: 0.8200\n",
            "Epoch 98/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.4011 - accuracy: 0.8200\n",
            "Epoch 99/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.3977 - accuracy: 0.8250\n",
            "Epoch 100/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.3949 - accuracy: 0.8300\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7fa5740cdb10>"
            ]
          },
          "metadata": {},
          "execution_count": 94
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "new_weights = model.get_weights()[0]"
      ],
      "metadata": {
        "id": "J6PwGDAvUGGg"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.optimizer.get_config()[\"learning_rate\"]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rgu2pr0lWdDu",
        "outputId": "e6d4d5b3-4d68-476d-f2c1-bdecb9aa22df"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.001"
            ]
          },
          "metadata": {},
          "execution_count": 71
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "gradient = (old_weights - new_weights)/ 0.001\n",
        "percent_change = abs(100*(old_weights - new_weights)/ old_weights)"
      ],
      "metadata": {
        "id": "dp5IJsE-WI7w"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "gradient"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Av5OspqLWl0r",
        "outputId": "4f3b4f6b-56e8-4271-9d9c-65703db844cc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 0.01093745,  0.0115037 ,  0.01388788, -0.01996755, -0.05242321,\n",
              "        -0.00369549,  0.01478195,  0.03504753,  0.02288818, -0.00494719],\n",
              "       [-0.02099573, -0.02273917, -0.02834201,  0.04476308,  0.09220838,\n",
              "         0.00709295, -0.02682209, -0.06717443, -0.05042553,  0.00888109]],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 73
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "percent_change"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YJsZpaeoWm-8",
        "outputId": "807dc184-605d-482f-8b17-96cdb9a95aaa"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.0072271 , 0.00210111, 0.00679461, 0.00370892, 0.36522916,\n",
              "        0.00057803, 0.00331642, 0.00498917, 0.00324947, 0.00184224],\n",
              "       [0.01097624, 0.00796773, 0.00666361, 0.01916067, 0.01916059,\n",
              "        0.00211133, 0.0059982 , 0.01164935, 0.01972343, 0.00127616]],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 74
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "old_weights"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5Uu6DITscBx5",
        "outputId": "e15e6498-5011-4db4-fe42-33ea2cf20f74"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-0.33058235,  0.292853  , -0.5027271 , -0.01332176,  0.4663965 ,\n",
              "        -0.08549386, -0.3767269 ,  0.02521479,  0.17957473,  0.09508276],\n",
              "       [-0.46751493, -0.5052102 ,  0.618305  , -0.41437078,  0.51485544,\n",
              "         0.45250946,  0.36829787,  0.66284555,  0.14777309,  0.04143274]],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 96
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "new_weights"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "72YfOy_-cEPj",
        "outputId": "3fb85c31-3033-4039-f30c-d12200df2b37"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 0.40518048,  0.6488755 , -0.985546  ,  0.6181405 ,  1.034089  ,\n",
              "        -0.7342173 , -0.5503599 ,  0.18725553,  0.81843793, -0.69256586],\n",
              "       [-1.5294187 , -1.299696  ,  1.6635121 , -1.4268929 , -0.56944025,\n",
              "         1.5625011 ,  0.9726272 ,  0.81920713, -0.9686132 ,  1.2659652 ]],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 97
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model = Sequential()\n",
        "\n",
        "model.add(Dense(10,activation='relu',input_dim=2))\n",
        "model.add(Dense(10,activation='relu'))\n",
        "model.add(Dense(10,activation='relu'))\n",
        "model.add(Dense(10,activation='relu'))\n",
        "model.add(Dense(10,activation='relu'))\n",
        "model.add(Dense(10,activation='relu'))\n",
        "model.add(Dense(10,activation='relu'))\n",
        "model.add(Dense(10,activation='relu'))\n",
        "model.add(Dense(10,activation='relu'))\n",
        "model.add(Dense(10,activation='relu'))\n",
        "model.add(Dense(10,activation='relu'))\n",
        "model.add(Dense(1, activation='sigmoid'))"
      ],
      "metadata": {
        "id": "nihtN5tYWqRR"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])"
      ],
      "metadata": {
        "id": "btOwuajlakTh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "old_weights = model.get_weights()[0]"
      ],
      "metadata": {
        "id": "fqGsgRFOanlJ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=42)"
      ],
      "metadata": {
        "id": "Wsafrolvap9R"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.fit(X_train, y_train, epochs = 100)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nwFILWxLasTl",
        "outputId": "eb1e6260-4a7e-4e19-f3ef-607aee952468"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/100\n",
            "7/7 [==============================] - 2s 6ms/step - loss: 0.6932 - accuracy: 0.4900\n",
            "Epoch 2/100\n",
            "7/7 [==============================] - 0s 6ms/step - loss: 0.6925 - accuracy: 0.5200\n",
            "Epoch 3/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.6914 - accuracy: 0.6950\n",
            "Epoch 4/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6897 - accuracy: 0.7950\n",
            "Epoch 5/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6872 - accuracy: 0.8300\n",
            "Epoch 6/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.6835 - accuracy: 0.8350\n",
            "Epoch 7/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6788 - accuracy: 0.8550\n",
            "Epoch 8/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.6725 - accuracy: 0.8750\n",
            "Epoch 9/100\n",
            "7/7 [==============================] - 0s 5ms/step - loss: 0.6635 - accuracy: 0.8850\n",
            "Epoch 10/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.6516 - accuracy: 0.8900\n",
            "Epoch 11/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.6352 - accuracy: 0.9100\n",
            "Epoch 12/100\n",
            "7/7 [==============================] - 0s 5ms/step - loss: 0.6122 - accuracy: 0.9100\n",
            "Epoch 13/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.5817 - accuracy: 0.9100\n",
            "Epoch 14/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.5394 - accuracy: 0.9100\n",
            "Epoch 15/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.5003 - accuracy: 0.9100\n",
            "Epoch 16/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.4629 - accuracy: 0.9100\n",
            "Epoch 17/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.4305 - accuracy: 0.9250\n",
            "Epoch 18/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.3991 - accuracy: 0.9300\n",
            "Epoch 19/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.3661 - accuracy: 0.9250\n",
            "Epoch 20/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.3398 - accuracy: 0.9300\n",
            "Epoch 21/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.3108 - accuracy: 0.9300\n",
            "Epoch 22/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.2804 - accuracy: 0.9250\n",
            "Epoch 23/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.2527 - accuracy: 0.9400\n",
            "Epoch 24/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.2123 - accuracy: 0.9450\n",
            "Epoch 25/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.2030 - accuracy: 0.9250\n",
            "Epoch 26/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.1809 - accuracy: 0.9450\n",
            "Epoch 27/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.1642 - accuracy: 0.9500\n",
            "Epoch 28/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.1532 - accuracy: 0.9500\n",
            "Epoch 29/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.1352 - accuracy: 0.9550\n",
            "Epoch 30/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.1292 - accuracy: 0.9450\n",
            "Epoch 31/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.1181 - accuracy: 0.9550\n",
            "Epoch 32/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.1112 - accuracy: 0.9550\n",
            "Epoch 33/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.1119 - accuracy: 0.9550\n",
            "Epoch 34/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0955 - accuracy: 0.9600\n",
            "Epoch 35/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0880 - accuracy: 0.9600\n",
            "Epoch 36/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0745 - accuracy: 0.9750\n",
            "Epoch 37/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0844 - accuracy: 0.9600\n",
            "Epoch 38/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0657 - accuracy: 0.9750\n",
            "Epoch 39/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0632 - accuracy: 0.9750\n",
            "Epoch 40/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0663 - accuracy: 0.9650\n",
            "Epoch 41/100\n",
            "7/7 [==============================] - 0s 6ms/step - loss: 0.0620 - accuracy: 0.9750\n",
            "Epoch 42/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0479 - accuracy: 0.9800\n",
            "Epoch 43/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0429 - accuracy: 0.9800\n",
            "Epoch 44/100\n",
            "7/7 [==============================] - 0s 5ms/step - loss: 0.0375 - accuracy: 0.9850\n",
            "Epoch 45/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.0333 - accuracy: 0.9850\n",
            "Epoch 46/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.0298 - accuracy: 0.9850\n",
            "Epoch 47/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.0221 - accuracy: 1.0000\n",
            "Epoch 48/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.0187 - accuracy: 1.0000\n",
            "Epoch 49/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.0149 - accuracy: 1.0000\n",
            "Epoch 50/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0137 - accuracy: 1.0000\n",
            "Epoch 51/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.0109 - accuracy: 1.0000\n",
            "Epoch 52/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.0096 - accuracy: 1.0000\n",
            "Epoch 53/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0142 - accuracy: 1.0000\n",
            "Epoch 54/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0068 - accuracy: 1.0000\n",
            "Epoch 55/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0064 - accuracy: 1.0000\n",
            "Epoch 56/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0033 - accuracy: 1.0000\n",
            "Epoch 57/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0028 - accuracy: 1.0000\n",
            "Epoch 58/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0025 - accuracy: 1.0000\n",
            "Epoch 59/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0021 - accuracy: 1.0000\n",
            "Epoch 60/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 0.0019 - accuracy: 1.0000\n",
            "Epoch 61/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0017 - accuracy: 1.0000\n",
            "Epoch 62/100\n",
            "7/7 [==============================] - 0s 5ms/step - loss: 0.0015 - accuracy: 1.0000\n",
            "Epoch 63/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0014 - accuracy: 1.0000\n",
            "Epoch 64/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0013 - accuracy: 1.0000\n",
            "Epoch 65/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0012 - accuracy: 1.0000\n",
            "Epoch 66/100\n",
            "7/7 [==============================] - 0s 6ms/step - loss: 0.0012 - accuracy: 1.0000\n",
            "Epoch 67/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0011 - accuracy: 1.0000\n",
            "Epoch 68/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 0.0010 - accuracy: 1.0000\n",
            "Epoch 69/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 9.3356e-04 - accuracy: 1.0000\n",
            "Epoch 70/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 8.8658e-04 - accuracy: 1.0000\n",
            "Epoch 71/100\n",
            "7/7 [==============================] - 0s 5ms/step - loss: 8.4221e-04 - accuracy: 1.0000\n",
            "Epoch 72/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 8.0131e-04 - accuracy: 1.0000\n",
            "Epoch 73/100\n",
            "7/7 [==============================] - 0s 5ms/step - loss: 7.6855e-04 - accuracy: 1.0000\n",
            "Epoch 74/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 7.3906e-04 - accuracy: 1.0000\n",
            "Epoch 75/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 6.9945e-04 - accuracy: 1.0000\n",
            "Epoch 76/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 6.6928e-04 - accuracy: 1.0000\n",
            "Epoch 77/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 6.4307e-04 - accuracy: 1.0000\n",
            "Epoch 78/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 6.1745e-04 - accuracy: 1.0000\n",
            "Epoch 79/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 5.9475e-04 - accuracy: 1.0000\n",
            "Epoch 80/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 5.7381e-04 - accuracy: 1.0000\n",
            "Epoch 81/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 5.5051e-04 - accuracy: 1.0000\n",
            "Epoch 82/100\n",
            "7/7 [==============================] - 0s 6ms/step - loss: 5.2939e-04 - accuracy: 1.0000\n",
            "Epoch 83/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 5.1094e-04 - accuracy: 1.0000\n",
            "Epoch 84/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 4.9248e-04 - accuracy: 1.0000\n",
            "Epoch 85/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 4.7631e-04 - accuracy: 1.0000\n",
            "Epoch 86/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 4.5717e-04 - accuracy: 1.0000\n",
            "Epoch 87/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 4.4247e-04 - accuracy: 1.0000\n",
            "Epoch 88/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 4.2555e-04 - accuracy: 1.0000\n",
            "Epoch 89/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 4.1302e-04 - accuracy: 1.0000\n",
            "Epoch 90/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 3.9883e-04 - accuracy: 1.0000\n",
            "Epoch 91/100\n",
            "7/7 [==============================] - 0s 5ms/step - loss: 3.9086e-04 - accuracy: 1.0000\n",
            "Epoch 92/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 3.7613e-04 - accuracy: 1.0000\n",
            "Epoch 93/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 3.6513e-04 - accuracy: 1.0000\n",
            "Epoch 94/100\n",
            "7/7 [==============================] - 0s 5ms/step - loss: 3.5298e-04 - accuracy: 1.0000\n",
            "Epoch 95/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 3.4278e-04 - accuracy: 1.0000\n",
            "Epoch 96/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 3.3290e-04 - accuracy: 1.0000\n",
            "Epoch 97/100\n",
            "7/7 [==============================] - 0s 5ms/step - loss: 3.2261e-04 - accuracy: 1.0000\n",
            "Epoch 98/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 3.1428e-04 - accuracy: 1.0000\n",
            "Epoch 99/100\n",
            "7/7 [==============================] - 0s 4ms/step - loss: 3.0677e-04 - accuracy: 1.0000\n",
            "Epoch 100/100\n",
            "7/7 [==============================] - 0s 3ms/step - loss: 2.9866e-04 - accuracy: 1.0000\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.callbacks.History at 0x7fa57a363710>"
            ]
          },
          "metadata": {},
          "execution_count": 114
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "new_weights = model.get_weights()[0]"
      ],
      "metadata": {
        "id": "XwFICdC4aubt"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.optimizer.get_config()[\"learning_rate\"]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sVKywUK2axYB",
        "outputId": "af7ad4e8-9663-4d93-874a-502c7131f0d1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.001"
            ]
          },
          "metadata": {},
          "execution_count": 116
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "gradient = (old_weights - new_weights)/ 0.001\n",
        "percent_change = abs(100*(old_weights - new_weights)/ old_weights)"
      ],
      "metadata": {
        "id": "RFiBr47yazsA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "gradient"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ivFmVMQWa2Vw",
        "outputId": "051c5cf4-9d7e-42d0-84cc-67c25218a84f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 6.685495  , -3.4646986 , -5.3832526 ,  6.741613  , -6.8466363 ,\n",
              "         5.8019753 , -6.013676  ,  2.311021  ,  0.31654534,  1.327753  ],\n",
              "       [-5.534008  ,  2.8640625 ,  4.297107  , -6.487876  ,  6.334662  ,\n",
              "        -5.206227  ,  0.72604764, -4.4065375 , -1.9274204 , -6.5084095 ]],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 106
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "percent_change"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WGELmjRHa42h",
        "outputId": "06a77640-c519-4c46-8b0e-9d9672c9117a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1.1875801 ,  0.61953056,  1.5359294 ,  1.9122909 ,  1.6018827 ,\n",
              "         1.2191677 ,  5.3679805 ,  0.7799734 ,  0.15018749,  0.25854072],\n",
              "       [ 5.258102  ,  0.9604039 ,  1.1871884 ,  1.2909728 ,  1.2937336 ,\n",
              "         0.87578124, 26.933157  , 12.894475  ,  1.8821074 ,  2.2091355 ]],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 107
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "old_weights"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "twVvX_-qa70F",
        "outputId": "015d13c0-c631-4bae-bca9-73b5405d0ffe"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-0.06470567,  0.38120002,  0.2564506 ,  0.4796242 , -0.41284758,\n",
              "         0.12111282,  0.3640564 , -0.11997336, -0.7036418 , -0.12918228],\n",
              "       [ 0.4177161 , -0.646973  ,  0.5538413 ,  0.14968556, -0.1020028 ,\n",
              "        -0.25845426, -0.1095776 ,  0.22031492, -0.01622504, -0.3014395 ]],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 118
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "new_weights"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Cy_6H27scOnh",
        "outputId": "a57826c5-a358-4f47-8e83-48c08c0949a6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-0.00357266,  0.42682663,  0.36751232,  0.6510883 , -0.47869346,\n",
              "         0.22171871,  0.4815479 , -0.20610414, -0.73377985, -0.09355866],\n",
              "       [ 0.4225647 , -0.76475453,  0.56343955,  0.08604728, -0.1019215 ,\n",
              "        -0.35053518, -0.19012658,  0.28164935,  0.21199906, -0.29813236]],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 119
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "BlsT6KOCcQho"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}