Add motivation about duals to chapter 5

src/chapters/chapter_5/chapter_1-5.tex

@@ -1,10 +1,86 @@
-\section{Formulating duals}
+\section{Linear programming duals}
 
 In this chapter, we will discuss the notion of duality in the context of linear programming problems. Duality can be understood as a toolbox of technical results that makes available a collection of techniques and features that can be exploited to both further understand characteristics of the optimal solution and to devise specialised methods for solving linear programming problems.
 
 
 \subsection{Motivation}
 
+Let us return to the paint factory example. Recall its formulation, given by
+%
+\begin{align}
+    \maxi z &= 5x_1 + 4x_2 \tag{1} \\
+    \st 6x_1 + 4x_2 &\leq 24 \tag{2} \\
+    x_1 + 2x_2 &\leq 6 \tag{3} \\
+    x_2 - x_1 &\leq 1 \tag{4} \\
+    x_2 &\leq 2 \tag{5} \\
+    x_1, x_2 &\geq 0. \tag{6}
+\end{align}
+%
+To motivate our developments, suppose we do not have an LP solver at our disposal but have found (perhaps by trial and error) a feasible solution $(3, 1)$ with objective value $19$ and want to know how good of a solution this is. If we know nothing else about the problem, $19$ might be a very bad solution compared to the optimal solution. On the other hand, if we can somehow determine an upper bound for the solution, showing that this problem simply cannot have a solution with an objective value better than, say, $25$, we might consider the solution “good enough” even if we cannot show that it is optimal.
+
+Notice that the objective function (1) and the first resource constraint (2) imply
+%
+$$
+z = 5x_1 + 4x_2 \leq 6x_1 + 4x_2 \leq 24.
+$$
+%
+The first inequality comes from the fact that the coefficients of the nonnegative decision variables $x$ are greater or equal on the left-hand side. The first constraint (2) thus tells us that the optimal solution value cannot be more than $24$, because of the availability of resources. This upper bound might be higher than the optimal value, but it allows us to conclude that our solution with value $19$ cannot be improved by more than $26\%$, even if this theoretical upper bound could be reached. The second constraint (3) does not fulfil the requirement of coefficients being at least equal to those in the objective function, but multiplying the constraint by $5$ on both sides achieves just that. This gives us
+%
+$$
+z = 5x_1 + 4x_2 \leq 5x_1 + 10x_2 \leq 30,
+$$
+%
+which is a worse bound than $24$.
+
+Finally, we notice that two constraints can be combined to obtain a new constraint. Multiplying the first constraint by $0.75$ and the second by $0.5$, we can add the results together to obtain:
+$$
+(0.75 \times 6 + 0.5 \times 1)x_1 + (0.75 \times 4 + 0.5 \times 2)x_2 \leq 0.75 \times 24 + 0.5 \times 6,
+$$
+or
+$$
+5x_1 + 4x_2 \leq 21.
+$$
+Turns out, this upper bound $21$ is also the optimal solution.
+
+Let us next formalise these ideas. What we want to obtain is a constraint giving us an upper bound for the original problem, hereinafter referred to as the primal problem $P$. This constraint is obtained as a weighted combination of the primal constraints. Denote the weights as $p_i$ for each constraint $i \in \{1, \dots, m\}$. In the case of a primal problem of the form
+%
+\begin{align*}
+	(P) : \mini \ & c^\top x \\
+	\st 	  & Ax \leq b \\
+		  	  & x \geq 0,   
+\end{align*}
+%
+the weights $p$ result in a constraint:
+%
+$$
+\sum_{i=1}^m p_i \sum_{j=1}^n A_{ij} x_j \leq \sum_{i=1}^m p_i b_i,
+$$
+%
+where the coefficient of $x_j$ is $\sum_{i=1}^m p_i A_{ij}$. For this to give an upper bound of the objective function $\sum_{j=1}^n c_j x_j$, the condition
+%
+$$
+\sum_{i=1}^m p_i A_{ij} \geq c_j
+$$
+%
+must be satisfied for each $j \in \{1, \dots, n\}$. In matrix form, this becomes:
+%
+$$
+A^\top p \geq c.
+$$
+%
+Note that in order for us to be able to combine the constraints into one, the weights $p$ must be nonnegative, as negative weights would flip the inequalities in the corresponding primal constraints. Finally, we want the upper bound to be as tight as possible, so we minimise the right-hand side $b^\top p$ of this combined constraint, resulting in the dual problem:
+%
+\begin{align*}
+	(D) : \mini \ & b^\top p \\
+	\st 	  & A^\top p \geq c 
+			  & p \geq 0.
+\end{align*}
+
+
+
+\subsection{Formulating duals}
+
+
 Let us define the notation we will be using throughout the next chapters. As before, let $c \in \reals^n$, $b \in \reals^m$, $A \in \reals^{m \times n}$, and $P$ be the standard form linear programming problem
 %
 \begin{align*}