Machine Learning

Gadient Descent

Prerequisites to increase your understanding

• Programming knowledge
• Mathematical background
• Jupyter notebook knowledge

I will try to explain gradient descent a little in this assignment. I am going to abbreviate gradient descent as GD in this notebook now on. GD is an algorithm that is used to solve the machine learning problems such as linear regression. Simply, the goal of linear regression is to fit a line of corresponding given dataset or points. Let's consider that we want to fit a line for the points showed with the graphic below.

x (input)	y (output)
10.0	8.04
8.0	6.95
13.0	7.58
9.0	8.81
11.0	8.33
14.0	9.96
6.0	7.24
4.0	4.26
12.0	10.84
7.0	4.82
5.0	5.68

As you can guess, we are talking about a line and a line has an equation like $y = mx + b$. Here the coefficient of $x$, $m$ is the slope of the line and $b$ is y-intersect. We can simply say that the fittest line (the solution of linear regression) is strictly related to the values of $m$ and $b$. That's why it is obvious that we have to find the best pair of $m$ and $b$ values.

The common approach to finding the best values is to use a function named error function or cost function to measure how fit the line is.