8. Identification and Simultaneous-Equation Models.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "94972ed8-2865-4594-a83e-3d063c009aba",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sympy as sy\n",
    "from sympy.solvers.solveset import linsolve\n",
    "import pandas_datareader as pdr\n",
    "import datetime as dt\n",
    "import statsmodels.api as sm\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "896e1abc-97a6-43d5-b5a6-7fe361e70826",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Terminologies Of Identification </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ab2a93b-e85c-4d20-82ef-c1b42d63c705",
   "metadata": {},
   "source": [
    "Before discussing simultaneous-equation models, we should clarify some terminologies that are commonly seen in macroeconomic literature. \n",
    "\n",
    "The **identification** was originally a statistical term which describes a model with multiple sets of parameters, but generates the same distribution, therefore the parameters can't be identified by investigating the generated observations.\n",
    "\n",
    "In econometric research, identification concerns more about identifying the causality, especially in multiple equation models. For easy demonstration, below is a time series model of three equations. The subscript $t$ represents the time period.\n",
    "\n",
    "This is all called **structural model**, because they may portray the structure of economy or behaviors of an economic agent."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58042fff-4ced-450e-a8cd-618ef397d03c",
   "metadata": {},
   "source": [
    "\\begin{align}\n",
    "Y_{1t} &= \\quad\\qquad+\\beta_{12}Y_{2t}+\\beta_{13}Y_{3t} + \\gamma_{11} X_{1t}+ \\gamma_{12} X_{2t} +\\gamma_{13} X_{3t}+u_{1t}\\\\\n",
    "Y_{2t} &= \\beta_{11}Y_{1t}\\quad\\quad\\qquad+\\beta_{13}Y_{3t} + \\gamma_{21} X_{1t}+ \\gamma_{22} X_{2t} +\\gamma_{23} X_{3t}+u_{2t}\\\\\n",
    "Y_{2t} &= \\beta_{11}Y_{1t}+\\beta_{12}Y_{2t} +\\quad\\qquad+ \\gamma_{31} X_{1t}+ \\gamma_{32} X_{2t} +\\gamma_{33} X_{3t}+u_{2t}\\\\\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b3939de-b48e-412e-a9cb-322711ed5017",
   "metadata": {},
   "source": [
    "$Y_1$, $Y_2$ and $Y_3$ are **endogenous** variables, i.e. values are determined within the model; $X_1$, $X_2$ and $X_3$  are the **predetermined** variables, which are treated as non-stochastic; $u_1$, $u_2$ and $u_3$ are disturbance terms. Note that we have ceased using terms such as _dependent_ or _independent_ variables as in single equation model any more.\n",
    "\n",
    "The _predetermined_ variables are divided into two categories: **exogenous** (such as $X_{1t}, X_{1, t-1}$) and **lagged endogenous** (such as $Y_{1, t-1}$), which are not determined by the model in the current time period. But in practice, the researchers themselves should classify variables based on theoretical grounds or experience."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0aeaaf8f-a85a-4191-9af7-0dc72dec8080",
   "metadata": {},
   "source": [
    "A common macroeconomic practice is to convert the structural form equations into the **reduced form equations**, which express endogenous variables solely in terms of the predetermined variables and disturbance terms."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c21c211-1f79-4203-961e-ad1e04eba596",
   "metadata": {},
   "source": [
    "It would be considerably concise to define corresponding matrices as following\n",
    "$$\n",
    "\\boldsymbol{Y}_t = \n",
    "\\begin{bmatrix}\n",
    "Y_{1t} \\\\\n",
    "Y_{2t}\\\\\n",
    "Y_{3t}\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\boldsymbol{X}_t = \n",
    "\\begin{bmatrix}\n",
    "X_{1t} \\\\\n",
    "X_{2t}\\\\\n",
    "X_{3t}\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\boldsymbol{\\beta} = \n",
    "\\begin{bmatrix}\n",
    "0 & \\beta_{12}  & \\beta_{13}\\\\\n",
    "\\beta_{11} & 0 &\\beta_{13} \\\\\n",
    "\\beta_{11} & \\beta_{12} &0\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\boldsymbol{\\Gamma} = \n",
    "\\begin{bmatrix}\n",
    "\\gamma_{11} & \\gamma_{12}  & \\gamma_{13}\\\\\n",
    "\\gamma_{21} & \\gamma_{22} &\\gamma_{23} \\\\\n",
    "\\gamma_{31} & \\gamma_{32} &\\gamma_{33}\n",
    "\\end{bmatrix}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\boldsymbol{u}_t = \n",
    "\\begin{bmatrix}\n",
    "u_{1}\\\\\n",
    "u_{2} \\\\\n",
    "u_{3}\n",
    "\\end{bmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6491031b-c7ca-4902-8461-7b55d84f07d1",
   "metadata": {},
   "source": [
    "Then structural model in matrix form is\n",
    "$$\n",
    "\\boldsymbol{Y}_t = \\boldsymbol{\\beta}\\boldsymbol{Y}_t + \\boldsymbol{\\Gamma}\\boldsymbol{X}_t + \\boldsymbol{u}_t \n",
    "$$\n",
    "Rewrite as reduced-form\n",
    "$$\n",
    "\\boldsymbol{Y}_t = (\\boldsymbol{I}-\\boldsymbol{\\beta})^{-1}\\boldsymbol{\\Gamma}\\boldsymbol{X}_t+(\\boldsymbol{I}-\\boldsymbol{\\beta})^{-1}\\boldsymbol{u}_t\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "253380a6-253f-4257-b561-6d64434eba9a",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Keynesian Cross Model </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68bddd2b-b2a9-422d-97d6-d5c7da9a6214",
   "metadata": {},
   "source": [
    "Here is structural model, it describes the consumption of the economy\n",
    "$$\n",
    "\\begin{array}{ll}\n",
    "\\text { Consumption function: } & C_{t}=\\beta_{0}+\\beta_{1} Y_{t}+u_{t} \\quad 0<\\beta_{1}<1 \\\\\n",
    "\\text { Income identity: } & Y_{t}=C_{t}+I_{t}\n",
    "\\end{array}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f57341f-9e3e-4f27-a3df-9012d2ce241d",
   "metadata": {},
   "source": [
    "Join both equations and write as reduced form\n",
    "$$\n",
    "Y_t  = \\frac{\\beta_0}{1-\\beta_1}+\\frac{I_t}{1-\\beta_1}+\\frac{u_t}{1-\\beta_1}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8dad74a-caaa-4cdb-990d-3af565531161",
   "metadata": {},
   "source": [
    "Or write as\n",
    "$$\n",
    "Y_{t}=\\Pi_{0}+\\Pi_{1} I_{t}+w_{t}\\\\\n",
    "\\begin{aligned}\n",
    "\\Pi_{0} &=\\frac{\\beta_{0}}{1-\\beta_{1}} \\\\\n",
    "\\Pi_{1} &=\\frac{1}{1-\\beta_{1}} \\\\\n",
    "w_{t} &=\\frac{u_{t}}{1-\\beta_{1}}\n",
    "\\end{aligned}\n",
    "$$\n",
    "Substitute back to consumption function\n",
    "$$\n",
    "\\begin{gathered}\n",
    "C_{t}=\\Pi_{2}+\\Pi_{3} I_{t}+w_{t} \\\\\n",
    "\\Pi_{2}=\\frac{\\beta_{0}}{1-\\beta_{1}}\\\\\n",
    "\\Pi_{3}=\\frac{\\beta_{1}}{1-\\beta_{1}} \\\\\n",
    "w_{t}=\\frac{u_{t}}{1-\\beta_{1}}\n",
    "\\end{gathered}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa8a1388-9137-415c-b4d8-263f4cf5a3f6",
   "metadata": {},
   "source": [
    "Once we have the reduced-form, we can estimate reduced-form coefficients by OLS without suffering from inconsistency."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f8319e2-0a4d-4927-81c4-585cc3b95561",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Underidentification </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e96b3cca-94fc-4577-835e-9836fa0e0b1c",
   "metadata": {},
   "source": [
    "This is the basic demand and supply model\n",
    "\\begin{align}\n",
    "\\text{Demand function:}& \\quad Q_{t}^{d}=\\alpha_{0}+\\alpha_{1} P_{t}+u_{1 t} \\quad \\alpha_{1}<0\\\\\n",
    "\\text{Supply function:}& \\quad Q_{t}^{s}=\\beta_{0}+\\beta_{1} P_{t}+u_{2 t} \\quad \\beta_{1}>0\\\\\n",
    "\\text{Equilibrium condition:}& \\quad \\mathrm{Q}_{t}^{d}=\\mathrm{Q}_{t}^{s}\n",
    "\\end{align}\n",
    "If $u_{1t}$ changes, it means the demand will change due to factor other than $P_t$, which causes the demand curve shifting; so is $u_{2t}$, which causes supply curve shifting.\n",
    "\n",
    "However, notice that curve shifting will cause both $P$ and $Q$ change too, this means that $u_{1t}$ and $P_t$  are not independently distributed, so are $u_{2t}$ and $P_t$. This means we can't use OLS to estimate each equations independently.\n",
    "\n",
    "Equate the demand and supply\n",
    "$$\\alpha_{0}+\\alpha_{1} P_{t}+u_{1 t}=\\beta_{0}+\\beta_{1} P_{t}+u_{2 t}$$\n",
    "Solve for equilibrium $P_t$\n",
    "$$\n",
    "\\begin{gathered}\n",
    "P_{t}=\\Pi_{0}+v_{t} \\\\\n",
    "\\Pi_{0}=\\frac{\\beta_{0}-\\alpha_{0}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "v_{t}=\\frac{u_{2 t}-u_{1 t}}{\\alpha_{1}-\\beta_{1}}\n",
    "\\end{gathered}\n",
    "$$\n",
    "Substitute back to demand function to solve for equilibrium quantity\n",
    "$$\n",
    "\\begin{gathered}\n",
    "Q_{t}=\\Pi_{1}+w_{t} \\\\\n",
    "\\Pi_{1}=\\frac{\\alpha_{1} \\beta_{0}-\\alpha_{0} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "w_{t}=\\frac{\\alpha_{1} u_{2 t}-\\beta_{1} u_{1 t}}{\\alpha_{1}-\\beta_{1}}\n",
    "\\end{gathered}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39fdf2db-7e33-4e00-a677-0f819897a212",
   "metadata": {},
   "source": [
    "We obtain the reduced-form equations in terms of structural coefficients and disturbance term. We can estimate the reduced-form coefficients $\\Pi_{0}$  and $\\Pi_{1}$ by OLS. \n",
    "\n",
    "Let's say $\\Pi_{0}=3$  and $\\Pi_{1}=4$, but how do we pin down structural coefficients $\\alpha_0$, $\\alpha_1$, $\\beta_0$ and $\\beta_1$. \n",
    "$$\n",
    "\\Pi_{0}=\\frac{\\beta_{0}-\\alpha_{0}}{\\alpha_{1}-\\beta_{1}}\\\\\n",
    "\\Pi_{1}=\\frac{\\alpha_{1} \\beta_{0}-\\alpha_{0} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \n",
    "$$\n",
    "Two equations, but four unknowns, there are infinite amount of combination of them to satisfy the restriction, this is the exact question of identification - how to identify the structural coefficient even if we have reduced-form coefficients?\n",
    "\n",
    "The answer to this case: _we can't_.\n",
    "\n",
    "Because there is not enough information to solve the system, we call this **underidentification**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "631d5434-5880-4095-b84a-0f98eb66ffa8",
   "metadata": {},
   "source": [
    "Since the problem of underidentification is due to lack of information, how about we give it more information and test if it can be identified. \n",
    "\\begin{align}\n",
    "\\text{Demand function:}& \\quad Q^d_{t}=\\alpha_{0}+\\alpha_{1} P_{t}+\\alpha_{2} I_{t}+u_{1 t} \\quad &\\alpha_{1}<0, \\alpha_{2}>0\\\\\n",
    "\\text{Supply function:}&\\quad Q^s_{t}=\\beta_{0}+\\beta_{1} P_{t}+u_{2 t} \\quad &\\beta_{1}>0\n",
    "\\end{align}\n",
    "where $I$ is the income of the family."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96c3ddc9-e9b6-42e9-bcdb-c526677ae58a",
   "metadata": {},
   "source": [
    "Equate both function\n",
    "$$\n",
    "\\alpha_{0}+\\alpha_{1} P_{t}+\\alpha_{2} I_{t}+u_{1 t}=\\beta_{0}+\\beta_{1} P_{t}+u_{2 t}\n",
    "$$\n",
    "And solve for the equilibrium $P_t$\n",
    "$$\n",
    "P_{t}=\\Pi_{0}+\\Pi_{1} I_{t}+v_{t}\\\\\n",
    "\\begin{aligned}\n",
    "\\Pi_{0} &=\\frac{\\beta_{0}-\\alpha_{0}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "\\Pi_{1} &=-\\frac{\\alpha_{2}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "v_{t} &=\\frac{u_{2 t}-u_{1 t}}{\\alpha_{1}-\\beta_{1}}\n",
    "\\end{aligned}\n",
    "$$\n",
    "Substitute back to demand function to solve for $Q_t$\n",
    "$$\n",
    "\\begin{aligned}\n",
    "&Q_{t}=\\Pi_{2}+\\Pi_{3} I_{t}+w_{t} \\\\\n",
    "&\\Pi_{2}=\\frac{\\alpha_{1} \\beta_{0}-\\alpha_{0} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "&\\Pi_{3}=-\\frac{\\alpha_{2} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "&w_{t}=\\frac{\\alpha_{1} u_{2 t}-\\beta_{1} u_{1 t}}{\\alpha_{1}-\\beta_{1}}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b2e4f11a-74c7-4f82-bfee-8a391cc60b16",
   "metadata": {},
   "source": [
    "Let's assume $\\Pi_0 = 2$, $\\Pi_1 = 3$,  $\\Pi_2 = 4$ and  $\\Pi_3 = 5$, we have a system of four equations and five unknowns\n",
    "\\begin{align}\n",
    "2& =\\frac{\\beta_{0}-\\alpha_{0}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "3& =-\\frac{\\alpha_{2}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "4&=\\frac{\\alpha_{1} \\beta_{0}-\\alpha_{0} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "5&=-\\frac{\\alpha_{2} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \n",
    "\\end{align}\n",
    "One free variable, the system is still underidentified!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "038c1ccb-a234-4851-8842-6be9cd8fcf94",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Exact Identification </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7359f8e-a320-472a-a94f-ed5a0741f41e",
   "metadata": {},
   "source": [
    "We keep adding variables in the model, we will see if it can be identified\n",
    "\n",
    "\\begin{align}\n",
    "\\text{Demand function:}& \\quad Q_{t}=\\alpha_{0}+\\alpha_{1} P_{t}+\\alpha_{2} I_{t}+u_{1 t}\\\\\n",
    "\\text{Supply function:}& \\quad Q_{t}=\\beta_{0}+\\beta_{1} P_{t}+\\beta_{2} P_{t-1}+u_{2 t}\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c9a598f-9ada-4b5e-bbb4-47b709ec28c5",
   "metadata": {},
   "source": [
    "Equate both functions\n",
    "$$\n",
    "\\alpha_{0}+\\alpha_{1} P_{t}+\\alpha_{2} I_{t}+u_{1 t}=\\beta_{0}+\\beta_{1} P_{t}+\\beta_{2} P_{t-1}+u_{2 t}\n",
    "$$\n",
    "Solve for $P_t$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7924814-da2e-408c-afb0-7c22b7efeb55",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{gathered}\n",
    "P_{t}=\\Pi_{0}+\\Pi_{1} I_{t}+\\Pi_{2} P_{t-1}+v_{t} \\\\\n",
    "\\Pi_{0}=\\frac{\\beta_{0}-\\alpha_{0}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "\\Pi_{1}=-\\frac{\\alpha_{2}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "\\Pi_{2}=\\frac{\\beta_{2}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "v_{t}=\\frac{u_{2 t}-u_{1 t}}{\\alpha_{1}-\\beta_{1}}\n",
    "\\end{gathered}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96b5e38a-0539-49af-ac69-ea0ff96a43a8",
   "metadata": {},
   "source": [
    "Substitute back into demand function to solve for $Q_t$\n",
    "$$\n",
    "Q_{t}=\\Pi_{3}+\\Pi_{4} I_{t}+\\Pi_{5} P_{t-1}+w_{t}\\\\\n",
    "\\Pi_{3}=\\frac{\\alpha_{1} \\beta_{0}-\\alpha_{0} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "\\Pi_{4}=-\\frac{\\alpha_{2} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "\\Pi_{5}=\\frac{\\alpha_{1} \\beta_{2}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "w_{t}=\\frac{\\alpha_{1} u_{2 t}-\\beta_{1} u_{1 t}}{\\alpha_{1}-\\beta_{1}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e924ed5-7382-4822-9629-4994ceef41ef",
   "metadata": {},
   "source": [
    "Finally we obtain six equations and six unknowns. Assume $\\Pi_0 = 2$, $\\Pi_1 = -3$,  $\\Pi_2 = 4$,  $\\Pi_3 = 5$, $\\Pi_4 = 6$ and $\\Pi_5 = 7$, then solve the system, we obtain all the structural coefficients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "c83e2ec7-aec2-40b5-a28a-9d8439b1c3e9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(3/2, 7/4, 45/4, 9, -2, 15)]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a0, a1, a2, b0, b1, b2 = sy.symbols(\"a0, a1, a2, b0, b1, b2\")\n",
    "\n",
    "eq0 = sy.Eq(2 * (a1 - b1) - (b0 - a0), 0)\n",
    "eq1 = sy.Eq(-3 * (a1 - b1) + a2, 0)\n",
    "eq2 = sy.Eq(4 * (a1 - b1) - b2, 0)\n",
    "eq3 = sy.Eq(5 * (a1 - b1) - a1 * b0 + a0 * b1, 0)\n",
    "eq4 = sy.Eq(6 * (a1 - b1) + a2 * b1, 0)\n",
    "eq5 = sy.Eq(7 * (a1 - b1) - a1 * b2, 0)\n",
    "\n",
    "sy.solve([eq0, eq1, eq2, eq3, eq4, eq5], (a0, a1, a2, b0, b1, b2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e37ba24a-784e-4c25-995d-aa1e38a6e33f",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Over Identification </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd42fcd9-d3cc-48ca-b54d-b08ed8f34d66",
   "metadata": {},
   "source": [
    "What will happen if we keep adding more variables than necessary? The system will be overidentified, here is the example\n",
    "\\begin{align}\n",
    "\\text{Demand function:}& \\quad Q_{t}=\\alpha_{0}+\\alpha_{1} P_{t}+\\alpha_{2} I_{t}+\\alpha_{3} R_{t}+u_{1 t}\\\\\n",
    "\\text{Supply function:}& \\quad Q_{t}=\\beta_{0}+\\beta_{1} P_{t}+\\beta_{2} P_{t-1}+u_{2 t}\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39a57e22-508d-4e62-a858-1177de96afb9",
   "metadata": {},
   "source": [
    "Combine and solve for $P_t$ and $Q_t$\n",
    "$$\n",
    "\\begin{aligned}\n",
    "P_{t} &=\\Pi_{0}+\\Pi_{1} I_{t}+\\Pi_{2} R_{t}+\\Pi_{3} P_{t-1}+v_{t} \\\\\n",
    "Q_{t} &=\\Pi_{4}+\\Pi_{5} I_{t}+\\Pi_{6} R_{t}+\\Pi_{7} P_{t-1}+w_{t}\n",
    "\\end{aligned}\n",
    "$$\n",
    "where $R_t$ represents wealth.\n",
    "$$\n",
    "\\begin{align}\n",
    "\\Pi_{0}&=\\frac{\\beta_{0}-\\alpha_{0}}{\\alpha_{1}-\\beta_{1}} \\quad &\\Pi_{1}=-\\frac{\\alpha_{2}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "\\Pi_{2}&=-\\frac{\\alpha_{3}}{\\alpha_{1}-\\beta_{1}} \\quad &\\Pi_{3}=\\frac{\\beta_{2}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "\\Pi_{4}&=\\frac{\\alpha_{1} \\beta_{0}-\\alpha_{0} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \\quad &\\Pi_{5}=-\\frac{\\alpha_{2} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "\\Pi_{6}&=-\\frac{\\alpha_{3} \\beta_{1}}{\\alpha_{1}-\\beta_{1}} \\quad &\\Pi_{7}=\\frac{\\alpha_{1} \\beta_{2}}{\\alpha_{1}-\\beta_{1}} \\\\\n",
    "w_{t}&=\\frac{\\alpha_{1} u_{2 t}-\\beta_{1} u_{1 t}}{\\alpha_{1}-\\beta_{1}} \\quad &v_{t}=\\frac{u_{2 t}-u_{1 t}}{\\alpha_{1}-\\beta_{1}}\n",
    "\\end{align}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a903eefe-a841-47ae-ab88-abf9fd2b4ea9",
   "metadata": {},
   "source": [
    "Eight equations and seven unknowns, if you get lucky there can be a unique set of structural parameters, however mostly no unique solutions. Therefore, it doesn't mean more variables in the system is better.\n",
    "\n",
    "As below shows, $\\beta_0$ can't be identified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "cb80139a-9d12-4efe-9b84-279a0ec3e6a2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(b0, -7/3, 0, 0, b0, -7/3, 0)]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a0, a1, a2, a3, b0, b1, b2 = sy.symbols(\"a0, a1, a2, a3, b0, b1, b2\")\n",
    "\n",
    "eq0 = sy.Eq(2 * (a1 - b1) - (b0 - a0), 0)\n",
    "eq1 = sy.Eq(-3 * (a1 - b1) + a2, 0)\n",
    "eq2 = sy.Eq(4 * (a1 - b1) + a3, 0)\n",
    "eq3 = sy.Eq(5 * (a1 - b1) - b2, 0)\n",
    "eq4 = sy.Eq(6 * (a1 - b1) - a1 * b0 + a0 * b1, 0)\n",
    "eq5 = sy.Eq(7 * (a1 - b1) + a2 * b1, 0)\n",
    "eq6 = sy.Eq(8 * (a1 - b1) + a3 * b1, 0)\n",
    "eq7 = sy.Eq(9 * (a1 - b1) - a1 * b2, 0)\n",
    "\n",
    "sy.solve([eq0, eq1, eq2, eq3, eq4, eq5, eq6, eq7], (a0, a1, a2, a3, b0, b1, b2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da95e066-189e-4462-817d-a6a8d0b49a85",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Order Condition </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b8a8847-9bb3-4131-9e11-15ed29cec190",
   "metadata": {},
   "source": [
    "We will use an example to show how to perform order condition and rank condition.\n",
    "\n",
    "Consider an extended Keynesian model\n",
    "$$\n",
    "\\begin{array}{ll}\n",
    "\\text { Consumption function: } & C_{t} & =\\beta_{1}+\\beta_{2} Y_{t}-\\beta_{3} T_{t}+u_{1 t} \\\\\n",
    "\\text { Investment function: } & I_{t} & =\\alpha_{0}+\\alpha_{1} Y_{t-1}+u_{2 t} \\\\\n",
    "\\text { Taxation function: } & T_{t} & =\\gamma_{0}+\\gamma_{1} Y_{t}+u_{3 t} \\\\\n",
    "\\text { Income identity: } & Y_{t} & =C_{t}+I_{t}+G_{t}\n",
    "\\end{array}\n",
    "$$\n",
    "In the model the endogenous variables are $C_t$, $I_t$, $T_t$, and $Y_t$ and the predetermined variables are $G_t$ and $Y_{t-1}$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c929d8f-1df5-471e-b33b-c3c4dbcd494a",
   "metadata": {},
   "source": [
    "**Step 1:** Arrange all equations that variables stay on the left side, disturbance terms on the right side.\n",
    "$$\n",
    "\\begin{array}{ll}\n",
    "\\text { Consumption function: } & C_{t} -\\beta_{1}+\\beta_{2} Y_{t}+\\beta_{3} T_{t}=u_{1 t} \\\\\n",
    "\\text { Investment function: } & I_{t} -\\alpha_{0}-\\alpha_{1} Y_{t-1}=u_{2 t} \\\\\n",
    "\\text { Taxation function: } & T_{t} -\\gamma_{0}-\\gamma_{1} Y_{t}=u_{3 t} \\\\\n",
    "\\text { Income identity: } & Y_{t} -C_{t}-I_{t}-G_{t}=0\n",
    "\\end{array}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e92f53e5-15a5-4e65-819c-59c6b8821cfb",
   "metadata": {},
   "source": [
    "**Step 2:** Extract the coefficient matrix like below\n",
    "$$\n",
    "\\qquad\\quad\\text{ Coefficients of the Variables }\\\\\n",
    "\\begin{aligned}\n",
    "&\\begin{array}{cccccccccc}\n",
    " &1 & C_{t} & I_{t} & T_{t} & Y_{t} & Y_{t-1} & G_{t}\\\\\n",
    "\\text{Cons. Func.} & -\\beta_1 & 1 & 0 & \\beta_{3} & \\beta_2 & 0 & 0  \\\\\n",
    "\\text{Inv. Func.}  & -\\alpha_0 & 0 & 1 & 0 & 0 & -\\alpha_1 & 0\\\\\n",
    "\\text{Tax. Func.}  & -\\gamma_0 & 0 & 0 & 1 & -\\gamma_1 & 0 & 0  \\\\\n",
    "\\text{Inco. Id.} & 0 & -1 & -1 & 0 & 1 & 0 & -1 \\\\\n",
    "\\end{array}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcc010a6-b7f1-4368-bbd0-9e5776763bfd",
   "metadata": {},
   "source": [
    "**Step 3:** To identify each equation, we follow a rule called **Order Condition**.\n",
    "\n",
    "In a model of $M$ simultaneous equations, in order for an equation to be identified, the number of predetermined variables excluded from the equation must not be less than the number of endogenous variables included in that equation less 1, that is,\n",
    "$$\n",
    "K-k \\geq m-1\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "M&= \\text{number of endogenous variables in the model}\\\\\n",
    "m&= \\text{number of endogenous variables in a given equation}\\\\\n",
    "K&= \\text{number of predetermined variables in the model}\\\\\n",
    "k&= \\text{number of predetermined variables in a given equation}\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "If $K-k=m-1$, the equation is exact identified, but if $K-k>m-1$, it is overidentified.\n",
    "\n",
    "Write a table like below"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2c5edb3-3f60-4423-9993-017b1789e2d6",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{array}{cccc} \n",
    "& \\text { No. of Predetermined } & \\text { No. of Endogenous } & \\\\\n",
    "&\\text { Variables Excluded, } & \\text { Variables Included, } & \\\\\n",
    " & (K-k) & \\text { Less One, }(m-1) & \\text { Identified? } \\\\\n",
    "\\text{Cons. Func.} & 2-0=2 & 3-1=2 & \\text { Exactly } \\\\\n",
    "\\text{Inv. Func.} & 2-1=1 & 1-1=0 & \\text { Overidentified } \\\\\n",
    "\\text{Tax. Func.} & 2-0=2 & 2-1=1 & \\text { Overidentified } \\\\\n",
    "\\text{Inco. Id.} & 2-1=1 & 3-1=2 & \\text { Underidentified }\n",
    "\\end{array}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05e47557-3bf6-40ab-afbf-b1e8651ad587",
   "metadata": {},
   "source": [
    "# <font face=\"gotham\" color=\"purple\"> Limited Information Methods </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "895a3e41-d92b-45cb-9d85-6334a8526a5d",
   "metadata": {},
   "source": [
    "**Limited information methods** (LIM) are estimation methods that used on each equation in the system individually, totally ignoring restrictions on other equations. In contrast, **full information methods** (FIM) estimate all equation simultaneously, taking account of restrictions from each equation. \n",
    "\n",
    "However, in practice FIM are seldom used for a variety of reason. Let alone computation burden, of most importance is sensitivity to specification error, if any equation out of hundreds of suffer from specification error, the whole model's estimation could be biased.\n",
    "\n",
    "Here we only discuss LIM, specifically there three common methods:\n",
    "1. Ordinary Least Squares\n",
    "2. Indirect Least Squares\n",
    "3. Two-Stage Least Squares"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5718fb68-aaa3-4a5b-a946-d466683cd0bb",
   "metadata": {},
   "source": [
    "## <font face=\"gotham\" color=\"purple\"> Ordinary Least Squares </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf75a32b-1ecf-4de8-a4a1-c752a068e8ee",
   "metadata": {},
   "source": [
    "In general, we don't use OLS for system of simultaneous equations due to interdependence of disturbance term and endogenous variables, the estimation results are biased and inconsistent. \n",
    "\n",
    "However there is one scenario that OLS can be applied in the contexts of simultaneous equations. This is the case of **recursive** models, consider the following model.\n",
    "\n",
    "$$\n",
    "\\begin{array}{ll}\n",
    "Y_{1 t}&=\\beta_{11}                                      &+\\gamma_{11} X_{1 t}+\\gamma_{12} X_{2 t}+u_{1 t} \\\\\n",
    "Y_{2 t}&=\\beta_{21}+\\beta_{22} Y_{1 t}                   &+\\gamma_{21} X_{1 t}+\\gamma_{22} X_{2 t}+u_{2 t} \\\\\n",
    "Y_{3 t}&=\\beta_{31}+\\beta_{32} Y_{1 t}+\\beta_{33} Y_{2 t}&+\\gamma_{31} X_{1 t}+\\gamma_{32} X_{2 t}+u_{3 t}\n",
    "\\end{array}\n",
    "$$\n",
    "where $Y$ and $X$ are endogenous and exogenous variables respectively. Assumptions of disturbance terms \n",
    "$$\n",
    "\\operatorname{Cov}\\left(u_{1 t}, u_{2 t}\\right)=\\operatorname{Cov}\\left(u_{1 t}, u_{3 t}\\right)=\\operatorname{Cov}\\left(u_{2 t}, u_{3 t}\\right)=0\n",
    "$$\n",
    "which can be called **zero contemporaneous correlation**, if you calculate covariance matrix of the disturbance term vector $\\mathbb{u}$, it is expected to be a diagonal matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "820a45b0-f388-492b-92cc-b0ff6eb81c77",
   "metadata": {},
   "source": [
    "Now, if you pay close attention, you can see the first equation can actually be estimated by OLS, because all exogenous variables are on right-hand side. \n",
    "\n",
    "The equation two has $Y_{1t}$ on the right hand side, however the $u_{1t}$ from $Y_{1t}$ and $u_{2t}$ from $Y_{2t}$ has zero covariance, again it can be estimated by OLS. \n",
    "\n",
    "Similarly, the third equations can also be estimated by OLS, because $Y_{1t}$ and $Y_{2t}$ are predetermined in the previous two equations. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c500ee66-6d07-498f-82cb-9fb6e2b999a0",
   "metadata": {},
   "source": [
    "## <font face=\"gotham\" color=\"purple\">Indirect Least Squares </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "715480e2-f28e-4fa3-9f84-abed4efbec77",
   "metadata": {},
   "source": [
    "**Indirect Least Squares** is the method for obtaining structural coefficients in exact identified model. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73b85e62-a0ce-4e43-87b0-227331dbd208",
   "metadata": {},
   "source": [
    "\\begin{align}\n",
    "\\text{Demand function:}& \\quad Q_{t}=\\alpha_{0}+\\alpha_{1} P_{t}+\\alpha_{2} X_{t}+u_{1 t}\\\\\n",
    "\\text{Supply function:}& \\quad Q_{t}=\\beta_{0}+\\beta_{1} P_{t}+\\beta_{2} P_{t-1}+u_{2 t}\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3f8b0cc-0215-453f-a582-973653512563",
   "metadata": {},
   "source": [
    "$Q$ and $P$ has their conventional meaning, $X$ can be anything, for this case, we deem it as income. The reduced-form equations are\n",
    "$$\n",
    "\\begin{aligned}\n",
    "P_{t}&=\\Pi_{0}+\\Pi_{1} X_{t}+\\Pi_{2} P_{t-1}+v_{t} \\\\\n",
    "Q_{t}&=\\Pi_{3}+\\Pi_{4} X_{t}+\\Pi_{5} P_{t-1}+w_{t}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36a0dabb-cf75-42bc-b737-0d6e7e7dac5d",
   "metadata": {},
   "source": [
    "Now, let's try with real data, which is excerpt from <a href='https://www.govinfo.gov/app/collection/erp/'>_Economic Report of the President 2020_</a>.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "940c074a-c3c2-4366-85e0-975b17048fd3",
   "metadata": {},
   "source": [
    "(Example will be updated...)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1bfbdb9-0608-497f-9406-578828da375c",
   "metadata": {},
   "source": [
    "## <font face=\"gotham\" color=\"purple\">Two Stage Least Squares </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7735fd70-42ef-4a74-aacb-e60f9bbee3e1",
   "metadata": {},
   "source": [
    "Will be updated..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff5ff6fb-31a4-4a3e-8269-374714d58638",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}