Linear_Regression_in_Python.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Linear Regression in Python.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d4PoGWSXpYjc"
      },
      "source": [
        "# Linear Regression\n",
        "Linear regression is probably one of the most important and widely used regression techniques. It’s among the simplest regression methods. One of its main advantages is the ease of interpreting results."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xL9KDwRinzwL"
      },
      "source": [
        "## Problem Formulation\n",
        "When implementing linear regression of some dependent variable 𝑦 on the set of independent variables 𝐱 = (𝑥₁, …, 𝑥ᵣ), where 𝑟 is the number of predictors, you assume a linear relationship between 𝑦 and 𝐱: 𝑦 = 𝛽₀ + 𝛽₁𝑥₁ + ⋯ + 𝛽ᵣ𝑥ᵣ + 𝜀. This equation is the **regression equation**. 𝛽₀, 𝛽₁, …, 𝛽ᵣ are the **regression coefficients**, and 𝜀 is the **random error**.\n",
        "\n",
        "Linear regression calculates the **estimators** of the regression coefficients or simply the **predicted weights**, denoted with 𝑏₀, 𝑏₁, …, 𝑏ᵣ. They define the **estimated regression function** 𝑓(𝐱) = 𝑏₀ + 𝑏₁𝑥₁ + ⋯ + 𝑏ᵣ𝑥ᵣ. This function should capture the dependencies between the inputs and output sufficiently well.\n",
        "\n",
        "The **estimated or predicted response**, 𝑓(𝐱ᵢ), for each observation 𝑖 = 1, …, 𝑛, should be as close as possible to the corresponding **actual response** 𝑦ᵢ. The differences 𝑦ᵢ - 𝑓(𝐱ᵢ) for all observations 𝑖 = 1, …, 𝑛, are called the **residuals**. Regression is about determining the best predicted weights, that is the weights corresponding to the smallest residuals.\n",
        "\n",
        "To get the best weights, you usually **minimize the sum of squared residuals (SSR)** for all observations 𝑖 = 1, …, 𝑛: SSR = Σᵢ(𝑦ᵢ - 𝑓(𝐱ᵢ))². This approach is called the method of ordinary least squares."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9NWKaD17n5Rx"
      },
      "source": [
        "## Regression Performance\n",
        "The variation of actual responses 𝑦ᵢ, 𝑖 = 1, …, 𝑛, occurs partly due to the dependence on the predictors 𝐱ᵢ. However, there is also an additional inherent variance of the output.\n",
        "\n",
        "The **coefficient of determination**, denoted as 𝑅², tells you which amount of variation in 𝑦 can be explained by the dependence on 𝐱 using the particular regression model. Larger 𝑅² indicates a better fit and means that the model can better explain the variation of the output with different inputs.\n",
        "\n",
        "The value 𝑅² = 1 corresponds to SSR = 0, that is to the **perfect fit** since the values of predicted and actual responses fit completely to each other."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mtVYltntn7-Z"
      },
      "source": [
        "Simple Linear Regression\n",
        "Simple or single-variate linear regression is the simplest case of linear regression with a single independent variable, 𝐱 = 𝑥.\n",
        "\n",
        "The following figure illustrates simple linear regression:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AvdYN1DToEOi"
      },
      "source": [
        "![fig-lin-reg.jpeg](data:image/jpeg;base64,UklGRsAjAABXRUJQVlA4TLQjAAAvtQN4AFXhYf9/+Rw5F2ZmTjbMdGG8MOOFmTm5MCd7YWY8CjPnmJnDyd2Fk7swc7Jh6f88z+/5/Z/n+f/+8NWdp9nqyNrqmOwqZP2r0JF2qtBf/8JTTbOawgq52q3cuFpNs9UeWEeudqsjVys3W/lgdTSV5cbV0cpXbHW0CrlaPcV3quNpprIsjbY6Wh3Y1YTJlas9vgtnXYVWU4VcravQ6lesQludq5CrdRVaTbEKM4xCW52m2cpbWCEHpxiFXIVd+QortNVqpNWEjlbTuAq72SpkbTXNCV0dWa7C5K0chpkqtHIVGv0KuwrTKLTVThXaarcKrQ4sN0G7CpMVZiZXM1XIlRXaasPkyoHRVXxuXIVW07g6WbJchblyY00VWh3YVWjFIJIkJy0dcPLvhJyDAt7AoQEGkWw3WemAk8QJvRcFxMCPh/7bYmw7bMNoCBhrww5oyknbjeWXK4o/t4sV7cI4La4EIcQtcZmcOWfIhbBYBJy2BXJWSG6HC+RsCLxmLoivu2nA1C1zlgRKFoHn2hHiVjhTZg+YuixmjZBcBmeN1IlL5eyAUovh9LkRZM6PbpKrxXNrwKjb5FwJFjLnzCUimReem0IyT86bIOB0lZwDt47cOSHzFBaKZjKpRTp9g+E4bSCrxRRcAJxzz+XDb5vzRXkFRF0HHkW7AS6dDpJLYxMCRPHmUvNcCs2Xu3A8i0KINz8zVg+FtGjym8ueP2fLgmhywVSOgVZiLeNVPgM27yOgkOYNjt0Q10iDRcKO1uVbSM2ODkaDUhkuGNfHwrgJOEPabcEJza4ORkDRBNeLKjcPd/zxuCquF9/CnQH8W3L4z7YtKmLu6z7HbniK0RhncRELWLZ5dkP8wMBgEaXMJqV+q0Zqy1lkKQyz6PefQ0qykZl+0aVY6UyYGeeIiuCLhF4iuoDN5/lm3XQrlbfvm+/9MstCn48wMgt7PqvEHDp4eMSRVTxiXNk28n0rVbbv2wij7Jg4Mju2xhzAkwWvAp3cDK5Yy1kpqc0mZ0aZnjgyc8jZGifJYeFzSJw6koVyShw2h8jR0i+8IiLikcN5xXqSEyad82vldDkKehwlZ0EGWSJWGelQOEU4zz4FDpMex8/c4bxJscPlEDhSTo04qUNFT4RCR0DQbWBxNRwvPYYpdk3IOnktDD7RkXF7twihe8r+qpgFE0uKHiwP79Iy2kqJ+q3b0fRX9XIxoMVGqIzMwPSuh4NmidyTmsH5/Ch0zMyGOXHJVOPXxuDjzAe/BUM2N86dK+XHChwNE6s5Kjr1nQt8G4fMCXAFXBXTHjyBvN1xc4XMlYc/HW4WWYeaXR358Wk575nU+cp4+H44HK4byTFyEcyRTWBXruweob9k7WO7c76W4icEd2l5CrKVzn079Fb0XDhC0IDiVUZmAetOrxbOG74qvCQ4P0GOhJuEeIWMc87j5gG+ioqLhnPOT4yjY2Wc3wLDFrtBMJnQoc1zzisfKjcMP06OmjjOm+fbp6Qr5/yYOAm4rHvk/pK1Rj/OL5t8RV1bvlTFnLpBVz4VnvmgQBzKgJ5exby6gd4x0I7zvRwRxwffStK5wamtrFPOHoh4Oai4X855zMEQFGdpHX5kLUMdPXfxCwrRsu/8ewqIOk9k/T4OJNePxSzJyWXdI/eX7MbJ/Dfubvr+nPOMPK7Pi6xNjIOYehqUd9vsxfzBwm63uwM2+Qe8OOjlPzietm/LKwYLqLg5LP4BFe2A0adHLuw9nEyARRX8yusCLx7JaTK1JE7xPOTdI/aXE9t9vz2vlvdG4dyFdvOivqqXhLsojoiNazU35l0GVhmZxX2V2u5R3r54K0kPpebrwiQDWtQswjmfM6riOOcBSSvMkjXO+VoWQhG8alQuzgvOOU9/WSJl90j9JVthvW0kzQ5uEXR7Ls2H/fYSGyIWSrl63Db9sJXlOh4eU2rHxshr2eUMMXKvl0vGP6kE5yHlLBou0eJPqPD0XR6QE9qzYDfyzcdFzr1wvmjK5SchdI/QXxWXB3bcIZcMXt51+XviIb9zSkTc30Vxndj674mvCKzApfbEuI6WhY6V2bKO8Tpw/m84XMuinPtmjrSq1XcMs5glhS6BSl/XD3S2MdEsqD9po/OgcNE8T9x6VdPnC5T/n9rx4dhMethJc+U8fLvhhyRSdo/QX867YKLSP2BmUZnD53pxVb4na9K6rRAPjTMmNwPekzVjn63ImIUq8lc5eQ6DHjlrcR4wfUOsxzkP7tjjWKiUblHi7Rs2PG1a0uHXwNuPE9ti2pdoW7BdoR/Z5arkv0DO0+M3/1v5e+IZq3MiRfdI/eW8AT6xkhl79nBJWphr6/3PG/l62X+pfr32//t+t7cjRJw5SxvSBt7/vOM9yEZmDzvmNtQSMU7jb8zpht+KUv68psxzTOjq2teo34h+173fbnghzd/Y4V5VmUKIDdfrZzP7Gm3z3bY5MOcsU6YTl9nKb18omXfMZYLOB1cXf0sztRXtPxYiFg9NsLH9em2U5i0ZW3wn4f3PrkXuu91wKuI47x/ouOyN75L0C+17I0Z1/swSpgmnFkejafMh1vzLKpauwalv2cmgiJmD978r04RTVR9+pCTEekGlLP2DQqGWZq3s/HkRyxULzjBNCGIrzadAHkR846cNsXQSTrVc4fCIPcZ+CbdpwqmIt5oJETt262LpKZyaJ683BuIoBfta7g+n/uFieVlEPHYuhjRLd8GUex0VJ0Icpoi075/Nm9lPpYHGFxrkwhkQEUcdLsI02akAIrba0SRx9NLTOU9EDeqYpKcb3D+cMsNH7dOiw4SqPm+zB6TXBD9FC03wU5xr21Fp8n3/pu+B+CPV5ovlibDV4/lVqoxB1knzU6mIYJM6N34JvwTOQ8plqYr5za706mMJLdTHEs696E45EXGXH9LWNJnLl3Oep4+i9gj1jwHNdTechKg6XGodpuTqbVY0pld7ak6rXqJK55x5yadKLOJv7vNVLE+E0abAtuq1RzivnzO28R3Lr6jXMaHoHoGi+15FvwjFSZQlTA5obQkTyhomoyMiBnHeveHnWfjmG9Niia2yZLaBk1E7UTY83VijTLwNOvSdIz8rpxhi1ECrL2Ga7LbszK2o1x7hraMmLF9g0Lwy6nVMqLqnVsKkgaJfhOIkyhImidaWMKGsYdKgcis/v7acT4qy7+El9inBby+2NZc54DBBMmuJU7a9KDSsOeFLlOxdM4AWZefIz8op1qJwA8sTYbXqcRlvlVRMvfYI5zONdUDOR0QZlTomlN1TUHa/s7JfJTBZsxIm1tQw4cueSV5UJWplnHOeMErt8ZL654pWfCCZx0RlS5cLwZNiLZz3xEpE8p8p9YKknyl0nSM+K+fYrkyT4RAx6U92pV57hO+lXBDnnE8to1LHhLJ7ChTdl9Uw0ayECX0NE/7Gu5fZxBhc1naSmvmdhIImo3eUuTGylK268lOdAznn/4Cr5yHF2+0vS4b8M0W1c51yRgWrPitn+GFMF5xV/ROoao8k4HdKRpUj1zGh7J4MVfcVxUk0K2FCW8PkvaPGk3m9psqxKk78mLqhaK8UHo1bZOGkvHMoj8+5Fon8Z0pYZelnimrnRgxJD1Z9Vi5OvpS1RyguHoqAtnvUJUyUNUy0KmFC7DBRbtyG/J++5WT6xraJyiYUNPm4IWivdN4h55zXHmNlPq0ejPOcWao3q3aOc54erPqsnFBtP8RuIdQQUb+ZWMeEsntq+AgqpUieBbtpVsKEsobJ0OW6y9TBNItt9Km2snqDEwqa5PwT2iu5asZxntYqkQfm5JwP9NOqVDsnp/asnE7Pu/sfQbuFWEOkddS05QuMnFdBrY4JTfdUNVD0i1CcRFnCpL21JUwIT4hcwqTu5FymC/40575Nv/+AvCv+mKKgiTsqlNO1Lti7RusKUxdfmVxnFRSdU1B7Vs6liCeeGBEHtFfINUT4w5N/T6xWx4Sme6q8in4pi5MQSpgc0NoSJoQnRC5hMnlFLm9D1OUJvdO7c76XYdsoCpqsPrUqpUb4LadCobz5AziXXWmXRUbROQW1Z+VMyl1kGMSJKq7DzV42vYOA2k8jTUuY0PxoG0beF2JBk49L4da3Z74LHv/G8rtQNwLys3Iepf3bwIj4so/yD6y6i5W79TBYmU6cwl7GGowkaEyLaVJ9NfALbdpN/rzcak/bIjZXiwTys3ISufsWHAVxjNd7CsvDHDcGaH5L42qt58O7NvYQ7NKtIyK2eStt9/0Dv73x3Z/y9j0Qc3VoaRkMOBXytO0QMU/XtqYJpzKCRy+HmDf/NJbBgFOlguohJo308FVNE07t6itPhkzEGXYivQDgVJW/rImYb9pGuzJNOFXiMztHIfZu38QyGHBqG+/fFDF22fvIME04lTDeDhFxWP9oy2DAqRHX2xCxR5lVuU0TTg25tCEQcUfbu1/LYMCp3L36ILaqO6llMOyrAihrY6iXIklECpqUKzGERLS+V4moW20TL6GwiX4lS5rgpzgVXv7z9oeIU37nP5imHfZ4fpaWRVKxhSTNzzoVWxhYmp/10vx0q23iJRQ20a9kSX0s4Txwx6cUQmy4xMksg2GX+VL/JNBfd2t01/XfAHStbeKlKGyiScmSeolqvXMWNK/VGBGL30WEadp3hGIhilIkiSj73iw10iiDduWfmXOiPCGkjCOalCv5BdWqJw+fM3XA5PR0K0KRkCuacEIxE1ItE7UrxP5oW9ukqKywiZV1TShpU9jEseYuP3Yq4vCbyLIMhv1WPc7SDkgoFiIvRaI04NDxxXAnbYJLhg1ByjiiSbmS0dSqnsRHFStfoXdYuhWhSMgVTTipmAmhlokqUn+0rW3ipSxsolayhI42hU0cab4XwGiIUc9cwcc07QjUoFHe5eTgJTCZ+G/dSnU4z8jM9SycD42DEDKOaFKuZGlq/WrTcS2cbwPTrQhFQqhoIkMoZkKoZaKK0B+Na5t4KQub0Jcs4Sp1TawubOI4y7jjGcshhtXOtgyGnRecZWm+8mIhFHw452GDc84r4T0RMo5oUq6krWq/xuSc897pVoQiIVc04ZxQzIRQy0QVoT8a1zbxUhY2US1ZIi9sQvGJ7NYWNnGYDfDFIyMmfVx94r5/9v5mZbEQVXGc83o1ZN85rQkZRzQpV7Jcun5NnG5NKBJiRRPOCcVMCJ+vrorQH41rm3gpC5uoliwhFTZRqWtiZWETB9mb1/+4JMRBv3gAy2A4DgjFQqxByDiiSbmS/iD3i3RhRSgSckUTzgnFTLSgbW0TL2VhE9WSJYrCJuSbNShs4hArWrsFYr4Z7zjDNB0KpGIhr5eLGiHjiBXlSojI/VLcHIjpVoQiIVc04ZxQzEQL2tY28VIWNlErWaIsbEKmQWETnRkrgzDbGCEg1+39gknb7Pe7O/zilMVC5KVI6BAyjlhRrkQNsV/xWOwl6vTOO4IVoUjIFU04qZiJFrStbeKlKGxCUbJEobMKLQqb6MtoGYTtkqxNDI+YOvY8cZumI0HeSiiKhShKkdAhZByxolyJGop+7a+YFaFIyBVNOKmYiQY0rm3ipShsQlGyhFDYhEyLwib6MlwGYYbRPyBXpGkth3WvhbTU0lzzZrTaJl6tC5twLQub6Mt4GYSZRf+AXA65mCU2mrTrsD8SoT2j1Tbxal3YhGtQ2MQgjJdBmFn0C8g1uSwgl4NuZcXylms4bRVui81vaWQ9H55FewgaMIMwo+gVkKvVGfFjGv6f/K62/XoNmEGYTXQKyPV3UkAulyUDZhBmEZ0Cci1RHpALSi+i/YdVBuRybTJgBmGmc+9j2bHKgFwuLA7Tw7QOyHXi1PExTVcWB+r3xJoG5Aqboqhpmi4zDuB7sr5lJzMgZh7gK6uapquU4TIIM4lmAbn+WwrI5TpltAzCLDYNKSCXS5WxMggzWNuueRCxH2QBueBUyw65lAG54FTEW7VBxI4fKQXkcrwUQMRyQ+3kEyglIumD0bX+NHOvznFFnAK2EpDrT1q7TdNB83h+8evFDpTS/HSkc1wRp4BtBOT6Nykgl8PGl3PeudABOWXTkc5xRZwCxg/I1UcWkMvR85fYllMe2ixSERLiIVtSOz5ahGAi5H+At6E5slnaxhVxyISGhoYyC3VALseCv0rrKteV1BNKY4+xF055aLMIRUjIVI+PlrKf1h/ZLE3jithZRmrs4+5JCsjl8Kkel/EhmZNQH9oswoe1kqkeHy1FPzU4slmaxhVx8tD/WarRzQgBueynJv7r/ZMO35khhIiZpFevR4kxEqlVoz+0WYQPRidTPT5ain5ad2SzWpNqflgVV0TZzdE7OqMynuNNYxGb7nMbpmnaU4Els7u0/MteWxH9kb9wlQaF97lcAwnO+pTOWIf60GZxZRESMvXjo6XoJ7lNqXZks7ZSStnuV6O4Ik6oMXsjRg30r8qAXHb2iumJF+so895CBNRobay7dM/ZNIDm0GapXRAPj5bq8dFSVhsh2yfVkc2yPq4IoZsfN3OcUIg1ZQG57LO9eNYhNl9a6kGzZgZ7WDJ2ozm0WWo3Ew+PFt3x0XoW7KbyB2OoHdksjeKKELqZ80+cUA9SclemaZ+V/dVneZRqcWK9FSSf+XVGqy/0d01jKA9tFqkICfHwaKkeHy1lP8mlQqpSH9ksK+OKKLrpjgp12L4Q4+5irwV5PPkzhEhpJAmuIfcUQUEiUt66ePVpq8eXl77SY+IkXm90maaxMxSLf5apchb6kTzr8Hq9T4ter7fRsLEDlh9hBK/X+xKb7bG/x5T14A/LNGz4J9m4elnPCP30xfG8ivZgseFeqZWo6aVsA/dSdGDUIl73UCO8udcb2XgmRTdLpgZ4DdrG99pA6xKpfSsbaZTmb2mR1M02B0ONQcaCSU0DGf8Q+N8dAkTKT0vuuIZeey3YTqmQKaiPbJY1pURI3fy4FMftPVlOyYUQghepQLhiU4x0ZLOsKSVC6uY0qb7OoOet5kgQQbWEf2lJp2ZM20MwGVtMp3NcEadPxHCjIjoSHqxuBdG3xh8KEZAS7/Lar9cAAbkcBEPfbYPoe6pdxBTL3WfhBt+yk/dfrsuL7gG50kwHQc//LlIxvEKAECLmUXpN9W8BwiXJ3fpPSAG5NL4Z8j75pstHEgJywSn3S4xNF5BrMkuT7WsMk0Ke9o3lAbnoBsM1Ousp9lhGcDFCQC46rtEZxrG/Sv03ISAXNdfobB7ZDxoH5JL2/XO8MOoDj+z2YNhXVaYIUwbkch45v4NRos57EgJywanAccdRBuSCUwmhxQkBueCU3xIJAbng1JBL+ztCQC449WOkgFxw5DkszdhefoLJrQnIBZW/J3ZXqkEIyAWnmm+eEJALPOkXkItmD0FYY6SAXLBGPSAXlFIJyAWnqhIDcsGp7DHDlAG5QIyxAnKV9jXGX5psaQDCQAG5/qS820D/4gEgDBSQK9I0IY2BAnJBHb1ncgvs6DuTW6DCaP83sZ4zuQUqDNf0nMktOBXtPywi7nC8BNMEGDYekKupLCAX/NFlJrdAkA4zuQVIDP9xFd+ykxmUM7kFHmzoG1PbmdyCQtoF5BpdCsgFp9p2Vczklo19ODSA0CQg19t/itvmthDoiHirNoSZ3IJHmszkFkiyKiDXfctmcgsoWTGTW5XvybKF4BJtQK6JnzjCNEETXUCu3T+vzf9fcxCB9g/cJAXkSpcCcsGpjOd4U8JMbsGp6mP2Vs7kFqR6kEzEFg8lBeSCVZg54ePtyjTBVWITlS0EqHdZgsMihhsVYOUuMgwi1NE9INckUEffgFxf9xRwd7OOAblmmi6CdXcJDQ0NdV3TLSDXClsycgu5rukYkAv46BaQC/xoHpDrxaU/zg9OVZUF5BpUNpNbMEjTgFw/vSvTBEPaBeTS6iiyw9kX0iwgl1urLQRn7+7QLiAXNNIoIBdA0iQgF5z6san6KI8iO1SyJiDXkKYJmKgDcv22ZQvBqeabH0sWkCvBPmyh1+l32fpt+ykWFeEC577bDRtzJrdsfypWXNx8b28oowbkYsDhBndhy/jpafMhhtUuasgtxIJD++6qNsBOBjV0QC4WHEZ/l7Sq9V88CXFkIwfkYsJUrLieZYeHIZYzeEAuJkwPh4uZT4GRohD7QRaQy8iYMBUrLmVZK5TN5JYiIBeQXkRMNxMiNv7O5rawhZgwFSuuYpN9nSIgl+oWAs2H/cMkL0sIyEUBKn9P7H7MivKAXNZsIYB8T9ZiGGyz8oBctrWFGDAVKy5e7lV5esgCctnPLTTahv53Q3Z0X6MXn0kSku+nnUzRnRQBuezuFrKf/GP7Q4jx00cQzqSM5O8nBORyWs0TzBbiK/OFOJGaJBICcjmz1pI5nHiw35wVwlm0q8ebkBCQy7klpqwhxg3bi5OowUMRAnI5v/JPXSrp8YQzqMTqByIE5HKGJeNm8wgn0DT7/E1lQC4nmRszfZnPWBOuzGZtPySdEJDLaRYXW1uoyX6UTRQM/+lbhbB4SxjI3n+6YMvdEwJyOdMKh8Wpav+dWV36FmlflrB4S9jHrr+DNuKJJyYE5HKivfk0/5b5KRSTsCHd73k90YTFW8I+dnxvlXuqPIwyIJdzrRLOEEo9QQyeloTFW8I+9nrP0LSh75sQkMtJpknL7YkgLd4S5rHLny7o7luQEJDLlaD7EmsRF28J89jhTzwapFtHZUAu14LlBu3+E4iLt4Q06ylMamUN0ff5dIOK+fQj4+bNpD9+u2IPxOFr/4K9mMyTsg7TWERqINK/clUhRKLyilNxEf207YgBuVwLF7+6vbppwoKweEuYd7M9+nRB6j9LNSN49HLKgFyuB2W/s2CIkCgXb4kdfhirVA+aWhmQyyWhjic4zdI+gbB4S+zv74nZZFf9D04IyOWiMK5H1pIJi7fE/r4ni01qoiIglysB8/daMNqnCzIr1FpvWUAulzKjTbgyqyRnmKarmfH+ehdW3WXxWkWArNy9+oAn6tOdCVi5W6fIA3K5vrHqb3/XPyCXy5sdGQx3+bFTlQG54JRv6X6QB+Si/K8FoJRxxzMSAnLBqQG+eGRlQC44VfUrPy5JGZALAukQkMtKoPiFfIgBuawGiO/uCBx3HMTUDd+t2+rJvAFFCXdRHBHH2rwmAbkA0W+TAnKBI40CcgEkLQJywamXn2By5enOBJasDMgFmGgDcoUm6DAWwIYqINe4gZaxgFO/QArIBaCoAnIBKJV1OXIAKSAXlCIE5PpvKSAXnMoOD1OuyxGgwKS2lgIjIOJoTysF5IJTLVfYFDFVti5HQAObAnLJTncmODXZ142BWCgly22a8IFFAbkiTTDlfkzS6c4ET4z72VD3O9hmlac7E7Tx9/f31zcg149UG9Ggp2gUVInuNKz8dGcy7DcmSMlI/n7C6c4EePQMyAV79AvIBX10CsgFgDQOyDX1mNVN0wRBmgbketPnoNn3D+y+kHYBudo1oy8MBHPv7tAqINf0Pd2mCYq0Ccg12CCWwQBGWgXkgkfaBOSCSBoE5IIkRp9w5UFI63IE8Oi3Lkds+zgRgoto5bocsf1T5xpIkANyyY4iOwNOnWt4QPd3qSoDciUpA3Kx4NS5BhEUAblYcOpcQ4i/rImYTyUgFxNOnWsAgRQBuZhw6lwDCJp1OcKEU+caSO5isucCULDl5iFABWw+LHcvq0Dl74lf/vP2Z/WZvQhX78kyQEAu689bQAacOteuaToH5Co+XYTq2UbIouNE6PJHcbozrbAlC88RRJc1/QJyjVDAxzRBFPl0ZwrPpv7GBDP6BeSCU1XrkwJywamiteUBuTJME0751CGtyxEgpHVALjikXUCu5qZpwiGtTnemGpXcpgmKtFmXIxNI63IEGGm1LkfgkSbrcgROuSuVIQTkAkvWBeSCTHQBucaRBeSCTFasyxHgRL0uR+DUt+xkUOW6HIFQ5NOdaWTZuhyBU9nhhHU5Aqd8iAG54FTICodXrssROBXxVjMR1uUInJonrzcG4kSydTkCSWxlXY7EmKYJS2xlXY5EghPbWJcjcMq9KtK6HIFT0Z0I63IEHHzC6ttX9AQIIURkyWoF23cxLp0DcgGE5uvdXje5tyryFCGPUtCHFZqQ1uUITCgrxGQyPh4/Ibzhn8cEuyr5IJnKdTkCFYTS3ab0hxAi+U/e0fY1KNwCMVO2LkcgxHDhwuJVPLuycSX+daAoZUAuGPGR7SW+nkFsev5EBvnvMMRynkmX64DMCojTPX8iGqqlZNQL7QNygcnF0mz9yvPu/kdoAnIB4F00Ccg1oCwgF7Tw8YwohHeKCYRejLAuR9bhNk1AkZ1dx5M7u4sQw1WeLGSSggm6McK6HDFNUOGRtU5CRH7legsmNhe2xd36T0bRKiAXWO21oPe6HIFT7k8hrcsROBVCCsgFUGwiIBc0suYP3CwVVE8ZkAtO7eorD0AIyAWnqkwRpgzIBaneM0oZkAtWKdblCLjqpH4U2cH0YUtw2MtPMDm8csenFEKEVs1rNZYF5AJQlOtyBOjoGpCrgg/g3UXPgFyUE7EAZvQMyAV79AvIBX10CsgFp3wqkAJywamsTRBOdyYYpPG6HLH0DQppvC5HAJF263JEmxNWEtZoEJCrV27NvgdgjSYBueCUe1XygFx+lk0Lp6L9CQG54JR7H8tORRxHti5H4JJV63IENGm3Lkeg8Qtpui5HQPHdHXTrcgQUmZ7q1sgmrcsRaLQNz16o+RDX5Qg8KjmuoNSyQy5lQC6Q5P+dTzFF3fYJ6kjrcgQoVd79BKWmef/dL1eVYl2OgKX39nQbX4gmnl8geoqgIOFvmCYd+urX8YeWpoVX8UiP6u7JEu9dOvyLI0h/MZHRTlVWcLpotE9hEeHpImpNJnzDxycw3Ol4CE1qNZN98SK/upepflWI8AgC4536jsDk9XbvFeITpkoWCeFCiI8sSmC808wSlgR43v+tQkYct/SvytUiMeCpygpKsj0Zj1KxWv/LFUL9igFPn1Ko2mth8/NE+OYfn8CApyorVHmW2wvfyZACXi+EEEIIA988BGwx8sNgi4F/TwxcDPyeLChjuFOVFcwY7VRlBTMG+x7I4b8c/svhvxz+y+G/HP7L4b8c/svhv8VHFzNJr16PEgOu+iN/4SoNCu9zudBqHWXeW4iAGq2h1eZLS49r1gxarbeC5DO/DlqlNJIE1yCkWgTSlvLTkjuWE0IEsYXzOBgO1AbS/gq08i8t6dQMWvWt8YdCBKTEKz0FWziPg+GAbqDl7rNwg2/ZyfsvV0Bri3mUXlP9W4BYvNGm/R4MtkT7mV20UVYvdi1GaOK/3j/p8J0ZLHrpdPMEs2YLNfdIrQhrBkOUala3V1C0yljosAeDDdFhZhdtk/WLXYsNAktmd2n5l722wp6XTmA1C8IWYoXHTEtLi2DNYNxTSn3fJj0HUBkLHfZgsCE6zOyiTdJisWuxo5meeOa8dPh6Ay0IW4gVfKUOMGYwAgr2pRkLPfZgsJ2my1ya2V7TYrFrMcRePOtgzkunwOaFBWELMQVjBuM56kbSjIUOezDYED1mdtEm6LDYtVjxr0O/+iyPUi2ONS+dxVDQlJC2ECOst8zrPXEMYwbD/79H3GeNfU6qNhY67MFgQ/SY2UWboMNi12LF23k8nvwZrHnp/OpfBgsL1myhtv1/S5e+S9x9HFsGo3avr3uK6DqedZDGQpc9GJiklk3RfLFrMUPGPwT+d4cAxrx0yr9/pIS8hZjREmoEs2Uwwj1FhRCDrXd8wljYiytLs6kreix2LWY0XqQCW146MQWzOOeekrwsg7aQmBXd2PKyae/5AYvWnu4qY6H5Hgx2/y46LXYtVtyrFlteOs09iuZD2EJswZTB6KrwYCpjof0eDDZEh5ldtI3/u0n7xa7FEA9WtwJbXjq8udQ80zVfLoO2UESZZLa8bEp5soUQ3VaoNhYM24NBh5ldtA3aL3YtNhj6bhtE31PtIiZ7Xjqymxmzhd4quGh0/NetcC9sGYzx/V8v9y8U8IxIGAvW7cGgw8wu2gbtF7sWG/T87yIVwysEkF46bMGYLfQpD9UrZYUFVsaawXjzz1xvykMFksYCaFsO/+XwH1AZMl8zixW2+hYBrG2FfWLE5mOfQkCr9x5lJ/+alAywN7VPzQwVAKs+pguA9duxxaN+AV6V6jNu5NTTwuvX+s23LyseHltC61cZ6pk3IoS3cXGxBJeaAA==)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0gwFXFrbpEsD"
      },
      "source": [
        "The estimated regression function (black line) has the equation 𝑓(𝑥) = 𝑏₀ + 𝑏₁𝑥. Your goal is to calculate the optimal values of the predicted weights 𝑏₀ and 𝑏₁ that minimize SSR and determine the estimated regression function. The value of 𝑏₀, also called the **intercept**, shows the point where the estimated regression line crosses the 𝑦 axis. It is the value of the estimated response 𝑓(𝑥) for 𝑥 = 0. The value of 𝑏₁ determines the **slope** of the estimated regression line."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y681ePhirWWm"
      },
      "source": [
        "# Simple Linear Regression With scikit-learn"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VuOxQf65rYCe"
      },
      "source": [
        "There are **five basic steps** when you’re implementing linear regression:\n",
        "\n",
        "\n",
        "1.   Import the packages and classes you need.\n",
        "2.   Provide data to work with and eventually do appropriate transformations.\n",
        "3.   Create a regression model and fit it with existing data.\n",
        "Check the results of model fitting to know whether the model is satisfactory.\n",
        "4.   Apply the model for predictions.\n",
        "\n",
        "These steps are more or less general for most of the regression approaches and implementations."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8GO0muLfroed"
      },
      "source": [
        "## Step 1: Import packages and classes\n",
        "\n",
        "The first step is to import the package numpy and the class LinearRegression from sklearn.linear_model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Mt-L5gFArCCw"
      },
      "source": [
        "import numpy as np\n",
        "from sklearn.linear_model import LinearRegression"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fZCg4kBxrwOM"
      },
      "source": [
        "Now, you have all the functionalities you need to implement linear regression.\n",
        "\n",
        "The fundamental data type of NumPy is the array type called numpy.ndarray. The rest of this article uses the term array to refer to instances of the type numpy.ndarray. \n",
        "\n",
        "The class sklearn.linear_model.LinearRegression will be used to perform linear and polynomial regression and make predictions accordingly."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6zNpG5_mrw97"
      },
      "source": [
        "## Step 2: Provide data\n",
        "\n",
        "The second step is defining data to work with. The inputs (regressors, 𝑥) and output (predictor, 𝑦) should be arrays (the instances of the class numpy.ndarray) or similar objects. This is the simplest way of providing data for regression:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vieIUjfGry2M"
      },
      "source": [
        "x = np.array([5, 15, 25, 35, 45, 55]).reshape((-1, 1))\n",
        "y = np.array([5, 20, 14, 32, 22, 38])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dDTjgINFr0es"
      },
      "source": [
        "Now, you have two arrays: the input x and output y. You should call .reshape() on x because this array is required to be two-dimensional, or to be more precise, to have one column and as many rows as necessary. That’s exactly what the argument (-1, 1) of .reshape() specifies.\n",
        "\n",
        "This is how x and y look now:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8npyNmvTr2Wk",
        "outputId": "01d94f43-b4bf-46cd-f723-30d84d0dc0ef"
      },
      "source": [
        "x"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 5],\n",
              "       [15],\n",
              "       [25],\n",
              "       [35],\n",
              "       [45],\n",
              "       [55]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HQU-mLensBSF",
        "outputId": "81eb6128-288b-4039-b0ef-e0e39df3e045"
      },
      "source": [
        "y"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 5, 20, 14, 32, 22, 38])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6CqvMMUYsIid"
      },
      "source": [
        "As you can see, x has two dimensions, and x.shape is (6, 1), while y has a single dimension, and y.shape is (6,)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yH78X1zIsJP8"
      },
      "source": [
        "## Step 3: Create a model and fit it\n",
        "\n",
        "The next step is to create a linear regression model and fit it using the existing data.\n",
        "\n",
        "Let’s create an instance of the class LinearRegression, which will represent the regression model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KucSzwoLsLI8"
      },
      "source": [
        "model = LinearRegression()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SU4XJKuOsPE8"
      },
      "source": [
        "This statement creates the variable model as the instance of LinearRegression. You can provide several optional parameters to LinearRegression:\n",
        "\n",
        "fit_intercept is a Boolean (True by default) that decides whether to calculate the intercept 𝑏₀ (True) or consider it equal to zero (False).\n",
        "normalize is a Boolean (False by default) that decides whether to normalize the input variables (True) or not (False).\n",
        "copy_X is a Boolean (True by default) that decides whether to copy (True) or overwrite the input variables (False).\n",
        "n_jobs is an integer or None (default) and represents the number of jobs used in parallel computation. None usually means one job and -1 to use all processors.\n",
        "This example uses the default values of all parameters.\n",
        "\n",
        "It’s time to start using the model. First, you need to call .fit() on model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qGlt6ba8sPnE",
        "outputId": "735900fc-03cc-48e3-ef91-0bbb2d7ee711"
      },
      "source": [
        "model.fit(x, y)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yAOQW3z9sREV"
      },
      "source": [
        "With .fit(), you calculate the optimal values of the weights 𝑏₀ and 𝑏₁, using the existing input and output (x and y) as the arguments. In other words, .fit() fits the model. It returns self, which is the variable model itself. That’s why you can replace the last two statements with this one"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-CG4kJ8ksUo8"
      },
      "source": [
        "model = LinearRegression().fit(x, y)\n",
        "#This statement does the same thing as the previous two. It’s just shorter."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xNpQekWLsZmc"
      },
      "source": [
        "## Step 4: Get results\n",
        "\n",
        "Once you have your model fitted, you can get the results to check whether the model works satisfactorily and interpret it.\n",
        "\n",
        "You can obtain the coefficient of determination (𝑅²) with .score() called on model:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jmcLoa9Xsdlk",
        "outputId": "83738295-42af-47bf-addc-708e34832757"
      },
      "source": [
        "r_sq = model.score(x, y)\n",
        "print('coefficient of determination:', r_sq)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "coefficient of determination: 0.7158756137479542\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PniMlbaCsbdM"
      },
      "source": [
        "When you’re applying .score(), the arguments are also the predictor x and regressor y, and the return value is 𝑅².\n",
        "\n",
        "The attributes of model are .intercept_, which represents the coefficient, 𝑏₀ and .coef_, which represents 𝑏₁:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ykumDwIQshcc",
        "outputId": "41674e76-3bf3-42dc-f954-d1c49782177c"
      },
      "source": [
        "print('intercept:', model.intercept_)\n",
        "print('slope:', model.coef_)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "intercept: 5.633333333333329\n",
            "slope: [0.54]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iYSINrjGsnvE"
      },
      "source": [
        "The code above illustrates how to get 𝑏₀ and 𝑏₁. You can notice that .intercept_ is a scalar, while .coef_ is an array.\n",
        "\n",
        "The value 𝑏₀ = 5.63 (approximately) illustrates that your model predicts the response 5.63 when 𝑥 is zero. The value 𝑏₁ = 0.54 means that the predicted response rises by 0.54 when 𝑥 is increased by one.\n",
        "\n",
        "You should notice that you can provide y as a two-dimensional array as well. In this case, you’ll get a similar result. This is how it might look:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y7snZCqHsoaU",
        "outputId": "0c3a5bd2-968d-44f4-c227-401d920fc59c"
      },
      "source": [
        "new_model = LinearRegression().fit(x, y.reshape((-1, 1)))\n",
        "print('intercept:', new_model.intercept_)\n",
        "print('slope:', new_model.coef_)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "intercept: [5.63333333]\n",
            "slope: [[0.54]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-sh5chVWuE2M"
      },
      "source": [
        "As you can see, this example is very similar to the previous one, but in this case, .intercept_ is a one-dimensional array with the single element 𝑏₀, and .coef_ is a two-dimensional array with the single element 𝑏₁."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xWtEPxRHsxFk"
      },
      "source": [
        "## Step 5: Predict response\n",
        "\n",
        "Once there is a satisfactory model, you can use it for predictions with either existing or new data.\n",
        "\n",
        "To obtain the predicted response, use .predict():"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dNdIpD7wsxxl",
        "outputId": "f831b9f6-1dc6-4c21-8246-058d37357479"
      },
      "source": [
        "y_pred = model.predict(x)\n",
        "print('predicted response:', y_pred, sep='\\n')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "predicted response:\n",
            "[ 8.33333333 13.73333333 19.13333333 24.53333333 29.93333333 35.33333333]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z4asEh9hs1ks"
      },
      "source": [
        "When applying .predict(), you pass the regressor as the argument and get the corresponding predicted response.\n",
        "\n",
        "This is a nearly identical way to predict the response:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ISQ3_z0Ts2OE",
        "outputId": "147d81aa-571c-4ed7-ed68-6f01be1504fa"
      },
      "source": [
        "y_pred = model.intercept_ + model.coef_ * x\n",
        "print('predicted response:', y_pred, sep='\\n')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "predicted response:\n",
            "[[ 8.33333333]\n",
            " [13.73333333]\n",
            " [19.13333333]\n",
            " [24.53333333]\n",
            " [29.93333333]\n",
            " [35.33333333]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aUqRJ1JCs96M"
      },
      "source": [
        "In this case, you multiply each element of x with model.coef_ and add model.intercept_ to the product.\n",
        "\n",
        "The output here differs from the previous example only in dimensions. The predicted response is now a two-dimensional array, while in the previous case, it had one dimension.\n",
        "\n",
        "If you reduce the number of dimensions of x to one, these two approaches will yield the same result. You can do this by replacing x with x.reshape(-1), x.flatten(), or x.ravel() when multiplying it with model.coef_.\n",
        "\n",
        "In practice, regression models are often applied for forecasts. This means that you can use fitted models to calculate the outputs based on some other, new inputs:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SGKYSum_s9hF",
        "outputId": "5d655ac3-d42e-4f4b-eafa-51116c0df291"
      },
      "source": [
        "x_new = np.arange(5).reshape((-1, 1))\n",
        "print(x_new)\n",
        "y_new = model.predict(x_new)\n",
        "print(y_new)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0]\n",
            " [1]\n",
            " [2]\n",
            " [3]\n",
            " [4]]\n",
            "[5.63333333 6.17333333 6.71333333 7.25333333 7.79333333]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MJrmt8hKtGtU"
      },
      "source": [
        "Here .predict() is applied to the new regressor x_new and yields the response y_new. This example conveniently uses arange() from numpy to generate an array with the elements from 0 (inclusive) to 5 (exclusive), that is 0, 1, 2, 3, and 4.\n",
        "\n",
        "You can find more information about LinearRegression on\n",
        "[*Official documentation page.*](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9sK-nrg2pj8C"
      },
      "source": [
        "---\n",
        "This artical is made by Devansh Mistry.\n"
      ]
    }
  ]
}