aiam
/
NEandBGD-Video
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
main
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'main'
${ noResults }
NEandBGD-Video/main.ipynb

1205 lines
204 KiB
Plaintext
Raw Permalink Normal View History Unescape Escape

Files Ok
7 months ago
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Linear regression\n",
"\n",
"The linear regression is a training procedure based on a linear model. The model makes a prediction by simply computing a weighted sum of the input features, plus a constant term called the bias term (also called the intercept term):\n",
"\n",
"$$ \\hat{y}=\\theta_0 x_0 + \\theta_1 x_1 + \\theta_2 x_2 + \\cdots + \\theta_n x_n$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This can be writen more easy by using vector notation form for $m$ values. Therefore, the model will become:\n",
"\n",
"$$ \n",
" \\begin{bmatrix}\n",
" \\hat{y}^0 \\\\ \n",
" \\hat{y}^1\\\\\n",
" \\hat{y}^2\\\\\n",
" \\vdots \\\\\n",
" \\hat{y}^m\n",
" \\end{bmatrix}\n",
" =\n",
" \\begin{bmatrix}\n",
" 1 & x_0^0 & x_1^0 & \\cdots x_n^0\\\\\n",
" 1 & x_0^1 & x_1^1 & \\cdots x_n^1\\\\\n",
" \\vdots & \\vdots \\\\\n",
" 1 & x_0^m & x_1^m & \\cdots x_n^m\n",
" \\end{bmatrix}\n",
" \\begin{bmatrix}\n",
" \\theta_0 \\\\\n",
" \\theta_1 \\\\\n",
" \\theta_2 \\\\\n",
" \\vdots \\\\\n",
" \\theta_n\n",
" \\end{bmatrix}\n",
"$$\n",
"Resulting:\n",
"\n",
"$$\\hat{y}= h_\\theta(x) = x \\theta $$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Now that we have our mode, how do we train it?**\n",
"\n",
"Please, consider that training the model means adjusting the parameters to reduce the error or minimizing the cost function. The most common performance measure of a regression model is the Mean Square Error (MSE). Therefore, to train a Linear Regression model, you need to find the value of θ that minimizes the MSE:\n",
"\n",
"$$ MSE(X,h_\\theta) = \\frac{1}{m} \\sum_{i=1}^{m} \\left(\\hat{y}^{(i)}-y^{(i)} \\right)^2$$\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$ MSE(X,h_\\theta) = \\frac{1}{m} \\sum_{i=1}^{m} \\left( x^{(i)}\\theta-y^{(i)} \\right)^2$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$ MSE(X,h_\\theta) = \\frac{1}{m} \\left( x\\theta-y \\right)^T \\left( x\\theta-y \\right)$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# The normal equation\n",
"\n",
"To find the value of $\\theta$ that minimizes the cost function, there is a closed-form solution that gives the result directly. This is called the **Normal Equation**; and can be find it by derivating the *MSE* equation as a function of $\\theta$ and making it equals to zero:\n",
"\n",
"\n",
"$$\\hat{\\theta} = (X^T X)^{-1} X^{T} y $$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$ Temp = \\theta_0 + \\theta_1 * t $$"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>0</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>24.218</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>23.154</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>24.347</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>24.411</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>24.411</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>295</th>\n",
" <td>46.357</td>\n",
" </tr>\n",
" <tr>\n",
" <th>296</th>\n",
" <td>46.551</td>\n",
" </tr>\n",
" <tr>\n",
" <th>297</th>\n",
" <td>46.519</td>\n",
" </tr>\n",
" <tr>\n",
" <th>298</th>\n",
" <td>46.551</td>\n",
" </tr>\n",
" <tr>\n",
" <th>299</th>\n",
" <td>46.583</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>300 rows × 1 columns</p>\n",
"</div>"
],
"text/plain": [
" 0\n",
"0 24.218\n",
"1 23.154\n",
"2 24.347\n",
"3 24.411\n",
"4 24.411\n",
".. ...\n",
"295 46.357\n",
"296 46.551\n",
"297 46.519\n",
"298 46.551\n",
"299 46.583\n",
"\n",
"[300 rows x 1 columns]"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd\n",
"df = pd.read_csv('data.csv')\n",
"df "
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 1., 0.],\n",
" [ 1., 1.],\n",
" [ 1., 2.],\n",
" [ 1., 3.],\n",
" [ 1., 4.],\n",
" [ 1., 5.],\n",
" [ 1., 6.],\n",
" [ 1., 7.],\n",
" [ 1., 8.],\n",
" [ 1., 9.],\n",
" [ 1., 10.],\n",
" [ 1., 11.],\n",
" [ 1., 12.],\n",
" [ 1., 13.],\n",
" [ 1., 14.],\n",
" [ 1., 15.],\n",
" [ 1., 16.],\n",
" [ 1., 17.],\n",
" [ 1., 18.],\n",
" [ 1., 19.],\n",
" [ 1., 20.],\n",
" [ 1., 21.],\n",
" [ 1., 22.],\n",
" [ 1., 23.],\n",
" [ 1., 24.],\n",
" [ 1., 25.],\n",
" [ 1., 26.],\n",
" [ 1., 27.],\n",
" [ 1., 28.],\n",
" [ 1., 29.],\n",
" [ 1., 30.],\n",
" [ 1., 31.],\n",
" [ 1., 32.],\n",
" [ 1., 33.],\n",
" [ 1., 34.],\n",
" [ 1., 35.],\n",
" [ 1., 36.],\n",
" [ 1., 37.],\n",
" [ 1., 38.],\n",
" [ 1., 39.],\n",
" [ 1., 40.],\n",
" [ 1., 41.],\n",
" [ 1., 42.],\n",
" [ 1., 43.],\n",
" [ 1., 44.],\n",
" [ 1., 45.],\n",
" [ 1., 46.],\n",
" [ 1., 47.],\n",
" [ 1., 48.],\n",
" [ 1., 49.],\n",
" [ 1., 50.],\n",
" [ 1., 51.],\n",
" [ 1., 52.],\n",
" [ 1., 53.],\n",
" [ 1., 54.],\n",
" [ 1., 55.],\n",
" [ 1., 56.],\n",
" [ 1., 57.],\n",
" [ 1., 58.],\n",
" [ 1., 59.],\n",
" [ 1., 60.],\n",
" [ 1., 61.],\n",
" [ 1., 62.],\n",
" [ 1., 63.],\n",
" [ 1., 64.],\n",
" [ 1., 65.],\n",
" [ 1., 66.],\n",
" [ 1., 67.],\n",
" [ 1., 68.],\n",
" [ 1., 69.],\n",
" [ 1., 70.],\n",
" [ 1., 71.],\n",
" [ 1., 72.],\n",
" [ 1., 73.],\n",
" [ 1., 74.],\n",
" [ 1., 75.],\n",
" [ 1., 76.],\n",
" [ 1., 77.],\n",
" [ 1., 78.],\n",
" [ 1., 79.],\n",
" [ 1., 80.],\n",
" [ 1., 81.],\n",
" [ 1., 82.],\n",
" [ 1., 83.],\n",
" [ 1., 84.],\n",
" [ 1., 85.],\n",
" [ 1., 86.],\n",
" [ 1., 87.],\n",
" [ 1., 88.],\n",
" [ 1., 89.],\n",
" [ 1., 90.],\n",
" [ 1., 91.],\n",
" [ 1., 92.],\n",
" [ 1., 93.],\n",
" [ 1., 94.],\n",
" [ 1., 95.],\n",
" [ 1., 96.],\n",
" [ 1., 97.],\n",
" [ 1., 98.],\n",
" [ 1., 99.],\n",
" [ 1., 100.],\n",
" [ 1., 101.],\n",
" [ 1., 102.],\n",
" [ 1., 103.],\n",
" [ 1., 104.],\n",
" [ 1., 105.],\n",
" [ 1., 106.],\n",
" [ 1., 107.],\n",
" [ 1., 108.],\n",
" [ 1., 109.],\n",
" [ 1., 110.],\n",
" [ 1., 111.],\n",
" [ 1., 112.],\n",
" [ 1., 113.],\n",
" [ 1., 114.],\n",
" [ 1., 115.],\n",
" [ 1., 116.],\n",
" [ 1., 117.],\n",
" [ 1., 118.],\n",
" [ 1., 119.],\n",
" [ 1., 120.],\n",
" [ 1., 121.],\n",
" [ 1., 122.],\n",
" [ 1., 123.],\n",
" [ 1., 124.],\n",
" [ 1., 125.],\n",
" [ 1., 126.],\n",
" [ 1., 127.],\n",
" [ 1., 128.],\n",
" [ 1., 129.],\n",
" [ 1., 130.],\n",
" [ 1., 131.],\n",
" [ 1., 132.],\n",
" [ 1., 133.],\n",
" [ 1., 134.],\n",
" [ 1., 135.],\n",
" [ 1., 136.],\n",
" [ 1., 137.],\n",
" [ 1., 138.],\n",
" [ 1., 139.],\n",
" [ 1., 140.],\n",
" [ 1., 141.],\n",
" [ 1., 142.],\n",
" [ 1., 143.],\n",
" [ 1., 144.],\n",
" [ 1., 145.],\n",
" [ 1., 146.],\n",
" [ 1., 147.],\n",
" [ 1., 148.],\n",
" [ 1., 149.],\n",
" [ 1., 150.],\n",
" [ 1., 151.],\n",
" [ 1., 152.],\n",
" [ 1., 153.],\n",
" [ 1., 154.],\n",
" [ 1., 155.],\n",
" [ 1., 156.],\n",
" [ 1., 157.],\n",
" [ 1., 158.],\n",
" [ 1., 159.],\n",
" [ 1., 160.],\n",
" [ 1., 161.],\n",
" [ 1., 162.],\n",
" [ 1., 163.],\n",
" [ 1., 164.],\n",
" [ 1., 165.],\n",
" [ 1., 166.],\n",
" [ 1., 167.],\n",
" [ 1., 168.],\n",
" [ 1., 169.],\n",
" [ 1., 170.],\n",
" [ 1., 171.],\n",
" [ 1., 172.],\n",
" [ 1., 173.],\n",
" [ 1., 174.],\n",
" [ 1., 175.],\n",
" [ 1., 176.],\n",
" [ 1., 177.],\n",
" [ 1., 178.],\n",
" [ 1., 179.],\n",
" [ 1., 180.],\n",
" [ 1., 181.],\n",
" [ 1., 182.],\n",
" [ 1., 183.],\n",
" [ 1., 184.],\n",
" [ 1., 185.],\n",
" [ 1., 186.],\n",
" [ 1., 187.],\n",
" [ 1., 188.],\n",
" [ 1., 189.],\n",
" [ 1., 190.],\n",
" [ 1., 191.],\n",
" [ 1., 192.],\n",
" [ 1., 193.],\n",
" [ 1., 194.],\n",
" [ 1., 195.],\n",
" [ 1., 196.],\n",
" [ 1., 197.],\n",
" [ 1., 198.],\n",
" [ 1., 199.],\n",
" [ 1., 200.],\n",
" [ 1., 201.],\n",
" [ 1., 202.],\n",
" [ 1., 203.],\n",
" [ 1., 204.],\n",
" [ 1., 205.],\n",
" [ 1., 206.],\n",
" [ 1., 207.],\n",
" [ 1., 208.],\n",
" [ 1., 209.],\n",
" [ 1., 210.],\n",
" [ 1., 211.],\n",
" [ 1., 212.],\n",
" [ 1., 213.],\n",
" [ 1., 214.],\n",
" [ 1., 215.],\n",
" [ 1., 216.],\n",
" [ 1., 217.],\n",
" [ 1., 218.],\n",
" [ 1., 219.],\n",
" [ 1., 220.],\n",
" [ 1., 221.],\n",
" [ 1., 222.],\n",
" [ 1., 223.],\n",
" [ 1., 224.],\n",
" [ 1., 225.],\n",
" [ 1., 226.],\n",
" [ 1., 227.],\n",
" [ 1., 228.],\n",
" [ 1., 229.],\n",
" [ 1., 230.],\n",
" [ 1., 231.],\n",
" [ 1., 232.],\n",
" [ 1., 233.],\n",
" [ 1., 234.],\n",
" [ 1., 235.],\n",
" [ 1., 236.],\n",
" [ 1., 237.],\n",
" [ 1., 238.],\n",
" [ 1., 239.],\n",
" [ 1., 240.],\n",
" [ 1., 241.],\n",
" [ 1., 242.],\n",
" [ 1., 243.],\n",
" [ 1., 244.],\n",
" [ 1., 245.],\n",
" [ 1., 246.],\n",
" [ 1., 247.],\n",
" [ 1., 248.],\n",
" [ 1., 249.],\n",
" [ 1., 250.],\n",
" [ 1., 251.],\n",
" [ 1., 252.],\n",
" [ 1., 253.],\n",
" [ 1., 254.],\n",
" [ 1., 255.],\n",
" [ 1., 256.],\n",
" [ 1., 257.],\n",
" [ 1., 258.],\n",
" [ 1., 259.],\n",
" [ 1., 260.],\n",
" [ 1., 261.],\n",
" [ 1., 262.],\n",
" [ 1., 263.],\n",
" [ 1., 264.],\n",
" [ 1., 265.],\n",
" [ 1., 266.],\n",
" [ 1., 267.],\n",
" [ 1., 268.],\n",
" [ 1., 269.],\n",
" [ 1., 270.],\n",
" [ 1., 271.],\n",
" [ 1., 272.],\n",
" [ 1., 273.],\n",
" [ 1., 274.],\n",
" [ 1., 275.],\n",
" [ 1., 276.],\n",
" [ 1., 277.],\n",
" [ 1., 278.],\n",
" [ 1., 279.],\n",
" [ 1., 280.],\n",
" [ 1., 281.],\n",
" [ 1., 282.],\n",
" [ 1., 283.],\n",
" [ 1., 284.],\n",
" [ 1., 285.],\n",
" [ 1., 286.],\n",
" [ 1., 287.],\n",
" [ 1., 288.],\n",
" [ 1., 289.],\n",
" [ 1., 290.],\n",
" [ 1., 291.],\n",
" [ 1., 292.],\n",
" [ 1., 293.],\n",
" [ 1., 294.],\n",
" [ 1., 295.],\n",
" [ 1., 296.],\n",
" [ 1., 297.],\n",
" [ 1., 298.],\n",
" [ 1., 299.]])"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import numpy as np\n",
"y = df['0']\n",
"n = 300\n",
"t = np.linspace(0,n-1,n)\n",
"X = np.c_[np.ones(len(t)), t]\n",
"X"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([25.70275643, 0.07850281])"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"theta = np.linalg.inv(X.T.dot(X)).dot(X.T).dot(y)\n",
"theta "
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 1. , 0. ],\n",
" [ 1. , 15.78947368],\n",
" [ 1. , 31.57894737],\n",
" [ 1. , 47.36842105],\n",
" [ 1. , 63.15789474],\n",
" [ 1. , 78.94736842],\n",
" [ 1. , 94.73684211],\n",
" [ 1. , 110.52631579],\n",
" [ 1. , 126.31578947],\n",
" [ 1. , 142.10526316],\n",
" [ 1. , 157.89473684],\n",
" [ 1. , 173.68421053],\n",
" [ 1. , 189.47368421],\n",
" [ 1. , 205.26315789],\n",
" [ 1. , 221.05263158],\n",
" [ 1. , 236.84210526],\n",
" [ 1. , 252.63157895],\n",
" [ 1. , 268.42105263],\n",
" [ 1. , 284.21052632],\n",
" [ 1. , 300. ]])"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"Xnew1 = np.linspace(0,300, 20)\n",
"Xnew = np.c_[np.ones(len(Xnew1)), Xnew1]\n",
"Xnew"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAABXpElEQVR4nO3deVxU9f4/8NecUXBhMXBBGxhNcEmzxUxpMVILLf2VW2Z62wiltFwR0MosCxLUrKsm3rnaZoj3m3Wtqy2KlAmGlKmZXPCKeFLUTEFQQWfO749PMzIwwAzMBvN6Ph488CxzzmdOwLz7LO+3SlEUBUREREROIrm6AURERORZGHwQERGRUzH4ICIiIqdi8EFEREROxeCDiIiInIrBBxERETkVgw8iIiJyKgYfRERE5FQtXN2A6gwGA06cOAFfX1+oVCpXN4eIiIisoCgKLly4gC5dukCS6u7bcLvg48SJEwgODnZ1M4iIiKgBjh8/Do1GU+c5bhd8+Pr6AhCN9/Pzc3FriIiIyBqlpaUIDg42fY7Xxe2CD+NQi5+fH4MPIiKiJsaaKROccEpEREROZVPw8eqrr0KlUpl99erVy3T88uXLmDZtGgIDA+Hj44OxY8fi1KlTdm80ERERNV0293z06dMHJ0+eNH3t2rXLdGzWrFnYsmULNm3ahMzMTJw4cQJjxoyxa4OJiIioabN5zkeLFi0QFBRUY39JSQl0Oh02bNiAIUOGAADWrVuH3r17Izs7G4MGDWp8a/+iKAquXr0KvV5vt2uSZS1btoRarXZ1M4iIqBmxOfjIz89Hly5d0KpVK4SHhyMxMREhISHIzc3FlStXMGzYMNO5vXr1QkhICLKysmoNPioqKlBRUWHaLi0trfP+lZWVOHnyJC5evGhr06kBVCoVNBoNfHx8XN0UIiJqJmwKPgYOHIj169ejZ8+eOHnyJBYtWoR77rkHBw8eRHFxMby8vNCuXTuz13Tq1AnFxcW1XjMxMRGLFi2y6v4GgwFHjx6FWq1Gly5d4OXlxURkDqQoCs6cOQNZlhEWFsYeECIisgubgo8RI0aY/t2vXz8MHDgQWq0W6enpaN26dYMakJCQgNmzZ5u2jeuELamsrITBYEBwcDDatGnToPuRbTp06IDCwkJcuXKFwQcREdlFo5batmvXDj169EBBQQGCgoJQWVmJ8+fPm51z6tQpi3NEjLy9vU05PazN7VFf2layH/YsERGRvTXqU7ysrAxHjhxB586d0b9/f7Rs2RLbt283Hc/Ly0NRURHCw8Mb3VAiIiJqHmwKPubOnYvMzEwUFhZi9+7dGD16NNRqNSZOnAh/f39ERUVh9uzZyMjIQG5uLp5++mmEh4fbdaULERERNcLevcCQIeK7i9gUfMiyjIkTJ6Jnz5549NFHERgYiOzsbHTo0AEAsHz5cowcORJjx47F4MGDERQUhE8//dQhDSfLIiIiMHPmTKvPX79+fY1JwkRE1Ix98AGQkQF8+KHLmmDThNO0tLQ6j7dq1QorV67EypUrG9Uop9m7F5g3D1iyBLj9dle3hoiIyDGOHQP++ANQqYCNG8W+tDTgyScBRQHatwe0Wqc1x+0KyzlV1eiPwQcRETVXXbvW3HfmDNC//7VtRXFac5r+shFFAcrLrf/67Tdg1y7ghx9E1AcAn3witnftEsetvZYN/6EiIiLwwgsvYObMmbjuuuvQqVMnrF27FuXl5Xj66afh6+uL0NBQbN261fSazMxM3HHHHfD29kbnzp0RHx+Pq1evmo6Xl5fjiSeegI+PDzp37oylS5fWuG9FRQXmzp2L66+/Hm3btsXAgQOxc+fOBj9uIiJqgtatA6qvFDV+hrVoAXz0kVOb0/R7Pi5eBBqbffPMGeDuu21/XVkZ0Lat1ae///77mDdvHn788Uds3LgRzz33HDZv3ozRo0dj/vz5WL58Of72t7+hqKgI586dw4MPPoinnnoKH3zwAQ4fPozo6Gi0atUKr776KgAgNjYWmZmZ+Pzzz9GxY0fMnz8fP/30E2655RbTPadPn45Dhw4hLS0NXbp0webNmzF8+HAcOHAAYWFhtr9nIiJqOhQF+OIL4I03AIPB8jl79gC33ebsdrmXkpISBYBSUlJS49ilS5eUQ4cOKZcuXbq2s6xMUcTjdf5XWZnV7+vee+9V7r77btP21atXlbZt2yp/+9vfTPtOnjypAFCysrKU+fPnKz179lQMBoPp+MqVKxUfHx9Fr9crFy5cULy8vJT09HTT8bNnzyqtW7dWZsyYoSiKohw7dkxRq9XK77//btaWoUOHKgkJCYqiKMq6desUf3//Wttt8ZkTEZH7++03RRk+/NpnVmCg+C5J5t9zc+1yu7o+v6tr+j0fbdqIHghb7Ntnuadj1y6gSq+BVfe2Qb9+/Uz/VqvVCAwMxE033WTa16lTJwDA6dOn8dtvvyE8PNwsydddd92FsrIyyLKMc+fOobKyEgMHDjQdDwgIQM+ePU3bBw4cgF6vR48ePczaUVFRgcDAQJvaTkRETURJCfDaa8A77wBXrwItWwKzZgFPPw3cdx8QHAxERQE6HXD8ONCxo9Ob2PSDD5XKpqEPAIAxFbwkiW4o4/fWrW2/lg1atmxptq1Sqcz2GQMNQ21dYzYqKyuDWq1Gbm5ujdToLBRHRNTMGAxibsf8+cDp02LfyJHAsmWAcZi9sBDw8hKfnVOmAJWVgLe305va9CecNkTHjkBQkJjl+9574ntQkEuiv9r07t0bWVlZUKpMav3hhx/g6+sLjUaD7t27o2XLltizZ4/p+Llz5/Df//7XtH3rrbdCr9fj9OnTCA0NNfuqK+U9ERE1Mbt3A3fcATz7rAg8evYEtm4Ftmy5FngAItAw9qirVC4JPIDm0PPREBqN20R/tXn++efx9ttv44UXXsD06dORl5eHhQsXYvbs2ZAkCT4+PoiKikJsbCwCAwPRsWNHLFiwwKzuTY8ePTBp0iQ88cQTWLp0KW699VacOXMG27dvR79+/fDQQw+58B0SEVGj/f47EBcHfPyx2PbzAxYuBKZPF59xbsozgw/APNBwYfRXm+uvvx7/+c9/EBsbi5tvvhkBAQGIiorCSy+9ZDonOTkZZWVlGDVqFHx9fTFnzhyUlJSYXWfdunVYvHgx5syZg99//x3t27fHoEGDMHLkSGe/JSIispfLl4Hly8UqlvJy8Tn2zDNi+6/5g+5MpVTt13cDpaWl8Pf3R0lJSY0Kt5cvX8bRo0fRrVs3tGrVykUt9Cx85kREbkRRgH//G5g9G/jf/8S+8HAxudTFyTLr+vyuzjPnfBARETU1v/0GREYCjzwiAo8uXUSG7h9+cHngYSsGH0RERO6ketXZ8+eBmTOBm24CvvlGzOVISADy8oDJk69NIG1CPHfOBxERkTsy1h374APg55/F0tk//hDHHn4YWLoU6N7dtW1sJAYfRERErmap6uzq1SJJGACEhgIrVwIPPOC6NtoRgw8iIiJXs1R1tkohURQUNJvAA+CcDyIiItf75z9rVp01ckHVWUdjzwcREZGrKArw2WeiFos7VZ11MPZ8EBERucKvvwL33w+MGSOybnfoIPYbe0Bq6wlpBprvO2sCIiIiMHPmTFc3g4iInOncOeDFF4Gbbwa2bxcZtl96SVRWd/O6Y/bCYZcmYufOnbjvvvtw7tw5tGvXztXNISIiW+n1wNq1ItA4e1bsGz1aLJ3t1k1su3ndMXth8EFERORo330nejt++UVs9+kDrFgBDB1qfp6b1x2zF48edpFlGRkZGZBl2eH3Ki8vxxNPPAEfHx907twZS5cuNTv+4Ycf4vbbb4evry+CgoLw+OOP4/Tp0wCAwsJC3HfffQCA6667DiqVCk899RQAYNu2bbj77rvRrl07BAYGYuTIkThy5IjD3w8REVmhqAiYMAG4914ReLRrJ+qw7NtXM/DwIB4bfOh0Omi1WgwZMgRarRY6nc6h94uNjUVmZiY+//xzfP3119i5cyd++ukn0/ErV67g9ddfxy+//ILPPvsMhYWFpgAjODgY//d//wcAyMvLw8mTJ7FixQoAIqiZPXs
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"ypre = Xnew.dot(theta)\n",
"plt.plot(Xnew1, ypre, '*-r', label='model')\n",
"plt.plot(t,y, '.k', label='data')\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Polynomial model \n",
"$$ Temp = \\theta_0 + \\theta_1 * t + \\theta_2 * t^2$$"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([ 2.28848082e+01, 1.35240024e-01, -1.89756565e-04])"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"X = np.c_[np.ones(len(t)), t, t*t]\n",
"theta = np.linalg.inv(X.T.dot(X)).dot(X.T).dot(y)\n",
"theta"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAABOzklEQVR4nO3deXhU5d3/8feZgUQgJJiwBJwQ0KSgIioYMa4U0IjiYxWorbQuTQlU8JF9UevSWuEBFGgfQYJ5pIqI2JZaULGAIS4sQpCCIpjwI4YjqywZCBIgc35/TDJkyDohmZlkPq/rykUy52Ry5zAkH+7zvb+3YVmWhYiIiIif2AI9ABEREQktCh8iIiLiVwofIiIi4lcKHyIiIuJXCh8iIiLiVwofIiIi4lcKHyIiIuJXCh8iIiLiV00CPYDzuVwu9u7dS8uWLTEMI9DDERERkRqwLIvjx4/ToUMHbLaq5zaCLnzs3buXuLi4QA9DREREamHPnj04HI4qzwm68NGyZUvAPfjIyMgAj0ZERERqwul0EhcX5/k9XpWgCx+lt1oiIyMVPkRERBqYmpRMqOBURERE/ErhQ0RERPxK4UNERET8KuhqPmrCsizOnj1LcXFxoIfS6DVt2hS73R7oYYiISCPS4MLH6dOn2bdvHydPngz0UEKCYRg4HA4iIiICPRQREWkkGlT4cLlc7N69G7vdTocOHQgLC1MjsnpkWRaHDh3CNE0SExM1AyIiInWiQYWP06dP43K5iIuLo3nz5oEeTkho06YNeXl5nDlzRuFDRETqRIMsOK2ubavUHc0siYhIXdNvcREREfErhQ8RERHxK4WPRqZ3796MGjWqxucvWLCAVq1a1dt4REQkOJhOk8zdmZhOM9BDaVgFp3Vu0yaYMAGmTYPrrgv0aEREROqM6TRZm78WDNh6YCtTPpuCy3JhM2ykD0gntUdqwMYW2uHjjTcgMxPefFPhQ0REGo2MzRkMXTYUC6vcMZflYti/fktKYSyOW+4OwOgaw20Xy4LCwpq/ffMNfPYZfP45LF7sfo6333Z//Nln7uM1fS6r/F9qZXr37s3jjz/OqFGjuPjii2nXrh3z58+nsLCQRx99lJYtW5KQkMCHH37o+ZysrCyuv/56wsPDad++PZMmTeLs2bOe44WFhTz00ENERETQvn17XnrppXJft6ioiHHjxnHJJZfQokULevXqxZo1a2p9uUVEJLht/H5jpcGjVLEBuf/8Pz+OylvDn/k4eRIutPvmoUNw882+f96JE9CiRY1P/+tf/8qECRP44osveOedd/jd737H0qVLue+++3jyySeZOXMmv/71r8nPz+fo0aPcddddPPLII7zxxhvs2LGDoUOHctFFF/Hcc88BMH78eLKysnjvvfdo27YtTz75JJs3b+aaa67xfM2RI0eyfft2Fi9eTIcOHVi6dCl33nkn27ZtIzEx0ffvWUREAsZ0muQcziEiLIITp0+QGOP+OZ5zOIdE8yQfzR3L0C7fVhk8AGwuSFiaBUM2u/8j3bo1xMf741sAwLAsH/777gdOp5OoqCgKCgqIjIz0Onbq1Cl2795N586dueiii9wPFhZeePioLR/CR+/evSkuLubTTz8FoLi4mKioKO6//37eeOMNAPbv30/79u1Zt24dy5Yt4+9//zvffPONp9fGnDlzmDhxIgUFBZw8eZKYmBgWLlzI4MGDAThy5AgOh4O0tDRmzZpFfn4+l156Kfn5+XTo0MEzln79+nH99dfz4osvsmDBAkaNGsWxY8cqHHeF11xERPyiNGwkxiTyUe5HpC1Pw2W5vM4xMNxhwwIDsKprz2TB9JUwbp3hPYN/gXGgqt/f52v4Mx/Nm7tDgC+2bKl4puOzz6DMrEGNvrYPunfv7nnfbrcTExPDVVdd5XmsXbt2ABw8eJBvvvmG5ORkryZfN910EydOnMA0TY4ePcrp06fp1auX53h0dDRdunTxfLxt2zaKi4v5yU9+4jWOoqIiYmJifBq7iIj4j+k0mb1+Ni+vfxmX5cLAHRQqChaeWQ6Dyuc7LPdxmwumroRx60ofBJo0gQUL6vpbqFLDDx+G4dOtDwCaNXP/abOBy3Xuz2bNfH8uHzRt2tTrY8MwvB4rDRoul3eqra0TJ05gt9vJzs4u1xpdG8WJiASPsjMci79azISVE7xunVil0xo+MlwwZ6uDnu2uoTCqOQmvLsHhPO+kDRugR48L+wZ81PDDR220bQuxsRAXB6mpkJEBe/a4Hw8Sl19+OX//+9+xLMsTSj7//HNatmyJw+EgOjqapk2bsmHDBjp27AjA0aNH+fbbb7ntttsAuPbaaykuLubgwYPccsstAfteRETErezy186tOrP76G4+3vgO87/7B6562M1ibKs7Gb60ZCHD5s0wbUn5/3gHQGiGD4cD8vIgLMw9c5KWBqdPQ3h4oEfm8dhjjzFr1iwef/xxRo4cyc6dO3n22WcZM2YMNpuNiIgIUlNTGT9+PDExMbRt25annnrKa9+bn/zkJwwZMoSHHnqIl156iWuvvZZDhw6xevVqunfvzt13B2aJlYhIKKpq+WttZjWqY8PgidT55x4Iov94h2b4AO+gYRhBFTwALrnkEj744APGjx/P1VdfTXR0NKmpqTz99NOec6ZPn86JEye45557aNmyJWPHjqWgoMDreV5//XVeeOEFxo4dy/fff0/r1q254YYbGDBggL+/JRGR0FPSzNJ8fixDP656+atPXGCzGbiw3LnFKik0LantsLtg3rVP44h0nPucIPqPd8Nf7SL1StdcRMRb2foMR6TD62PAsxR299HdsOB1Or+9gk+H3MzYFp/Vyde3uSD92t+T0ieN3CO5JBS1gLvvJjchmhZ330fh+0tJyD2CY81md+Dwk9Ba7SIiIuInGZszPMtdDQz6/3AxK1ofxVUyo+FZ9lqqJTAUMD7zzErUlFFyctnns7lg/WuQtOxnEOk4N7Px9R4cpTMaI18MulKC8zX8DqciIiJ+YDpNrz4bFhYfxBzxBI/Sx8oxyvxZ2b0GFxglx2wYjEseR/7ofObfPA17SU2oHRvpX3UiyRVbvk4jPNwdPCAoSwnOp5kPERGRGsg5nFOuwZfPhaIl58+9ay49nS3I+++HwIDkPYBhkHuxRcLC93Hc1B+A1L7jSOl+H7kn8kmIScTR8pKgn9WoCYUPEREJeV51HN/u9xSJ5jiae2o5dhza4fOtk8p0bdOVpPYJJB0pWX0yzb36xLFnD8Rf5XWuo81lONpcdu6BBh48QOFDRERC3Iy1M5i4aqKnjmPsj9fQruhLJq5eg8uwvOs4yqwoqbHzzrcbdhKiEyAyeFaf+JtqPkREJHRs2gR9+sCmTZhOk+HLhjN+5XivOo4Zzb5k/O3gKinCKFfHUbZ2ozSTuM4ViNqw8eTVj7Pk31Esyb6MpyL6Yys5z27YmTdg3rlC0QZWq1FXNPMhIiKh4403IDOTjHcmMTTi48r7blQ3s2HAzBVwUz4UhhskHLbgg/fJvaQZCdEJ7nDRfzqEhTHYMBhesIfcA9+QEHuFd++NEHVBMx9Tp07FMAxGjRrleax3794YhuH1Nnz48Asdp4iIiM9Mp0nm+rcxP/vA3V78nXcwIyGt+eoLavhld8GgpleT9Myr9G59HY7msTjir6J3p94Vzmo4ouLo/ZM7FDxK1HrmY+PGjcybN89rp9ZSQ4cO5Q9/+IPn4+Y+7v4aKnr37s0111zDrFmzAj0UEZEGz/z0fXJmPk3i6Bdw3HK3V08OmwvSl0HqQcjpBK4L+K+3zbAx7+5XcDw3LORqNepKrS7/iRMnGDJkCPPnz+fiiy8ud7x58+bExsZ63qrrdCbVW7NmDYZhcOzYsUAPRUQk6MxYO4P41QPoc/UW4lffw1Orn/LqyeGyQdo9YEbCpvZU3m+jiskQA4O5d83lu1HfkXr98JCs1agrtQofI0aM4O6776Zfv34VHn/rrbdo3bo13bp1Y/LkyZw8ebLS5yoqKsLpdHq9iYiI1Mh33zH9nSfcRaMlWcBlWLz46YvlenK4bPDCLTDpdsrVdAzvOZw9v/iC6es
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"Xnew1 = np.linspace(0,300,50)\n",
"Xnew = np.c_[np.ones(len(Xnew1)), Xnew1, Xnew1*Xnew1]\n",
"ypred = Xnew.dot(theta)\n",
"plt.plot(Xnew1, ypred, '*-r', label='model')\n",
"plt.plot(t,y, '.g', label='data')\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Batch Gradient Descent\n",
"\n",
"$$\\theta_{new} = \\theta_{old}-\\eta \\nabla_{\\theta} $$\n",
"\n",
"$$\\nabla_{\\theta} = \\frac{2}{m} X^T (X \\theta -y) $$"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[24.218],\n",
" [23.154],\n",
" [24.347],\n",
" [24.411],\n",
" [24.411],\n",
" [24.347],\n",
" [24.314],\n",
" [24.347],\n",
" [24.347],\n",
" [23.896],\n",
" [24.476],\n",
" [24.637],\n",
" [24.669],\n",
" [24.669],\n",
" [25.056],\n",
" [25.088],\n",
" [24.991],\n",
" [25.088],\n",
" [25.217],\n",
" [25.281],\n",
" [25.313],\n",
" [25.668],\n",
" [25.668],\n",
" [25.636],\n",
" [26.022],\n",
" [25.926],\n",
" [19.126],\n",
" [26.248],\n",
" [26.248],\n",
" [26.055],\n",
" [25.152],\n",
" [26.699],\n",
" [26.989],\n",
" [26.957],\n",
" [27.021],\n",
" [27.118],\n",
" [27.247],\n",
" [27.344],\n",
" [27.666],\n",
" [27.183],\n",
" [27.795],\n",
" [27.892],\n",
" [28.021],\n",
" [28.311],\n",
" [28.214],\n",
" [28.504],\n",
" [28.536],\n",
" [28.762],\n",
" [28.826],\n",
" [28.858],\n",
" [29.245],\n",
" [29.181],\n",
" [29.374],\n",
" [29.6 ],\n",
" [29.567],\n",
" [29.793],\n",
" [29.761],\n",
" [29.89 ],\n",
" [30.147],\n",
" [30.147],\n",
" [30.438],\n",
" [30.599],\n",
" [30.728],\n",
" [30.856],\n",
" [30.76 ],\n",
" [31.018],\n",
" [31.114],\n",
" [31.34 ],\n",
" [31.533],\n",
" [31.501],\n",
" [31.727],\n",
" [31.469],\n",
" [32.017],\n",
" [32.081],\n",
" [32.113],\n",
" [32.5 ],\n",
" [32.403],\n",
" [32.403],\n",
" [32.693],\n",
" [32.726],\n",
" [32.887],\n",
" [33.016],\n",
" [33.048],\n",
" [33.08 ],\n",
" [33.37 ],\n",
" [33.37 ],\n",
" [33.499],\n",
" [33.725],\n",
" [33.789],\n",
" [33.821],\n",
" [34.047],\n",
" [34.079],\n",
" [34.144],\n",
" [34.305],\n",
" [34.434],\n",
" [34.434],\n",
" [34.659],\n",
" [34.756],\n",
" [34.659],\n",
" [34.691],\n",
" [34.917],\n",
" [34.981],\n",
" [34.981],\n",
" [35.271],\n",
" [35.4 ],\n",
" [35.336],\n",
" [35.239],\n",
" [35.594],\n",
" [35.626],\n",
" [35.819],\n",
" [26.796],\n",
" [35.948],\n",
" [27.408],\n",
" [36.174],\n",
" [35.304],\n",
" [36.271],\n",
" [36.528],\n",
" [36.561],\n",
" [36.689],\n",
" [36.657],\n",
" [36.979],\n",
" [36.979],\n",
" [37.044],\n",
" [37.205],\n",
" [37.173],\n",
" [37.237],\n",
" [37.205],\n",
" [37.302],\n",
" [37.656],\n",
" [37.56 ],\n",
" [37.592],\n",
" [37.882],\n",
" [37.882],\n",
" [37.817],\n",
" [38.043],\n",
" [37.173],\n",
" [38.269],\n",
" [38.365],\n",
" [38.397],\n",
" [38.591],\n",
" [33.016],\n",
" [26.022],\n",
" [38.913],\n",
" [38.945],\n",
" [38.913],\n",
" [38.945],\n",
" [38.945],\n",
" [39.235],\n",
" [39.203],\n",
" [39.268],\n",
" [39.3 ],\n",
" [39.493],\n",
" [39.042],\n",
" [39.59 ],\n",
" [39.622],\n",
" [39.654],\n",
" [39.815],\n",
" [39.88 ],\n",
" [39.912],\n",
" [39.912],\n",
" [40.009],\n",
" [40.009],\n",
" [40.234],\n",
" [40.234],\n",
" [40.234],\n",
" [40.363],\n",
" [40.524],\n",
" [40.524],\n",
" [40.557],\n",
" [40.557],\n",
" [40.653],\n",
" [40.814],\n",
" [40.557],\n",
" [40.911],\n",
" [40.879],\n",
" [41.072],\n",
" [41.169],\n",
" [41.104],\n",
" [41.072],\n",
" [41.104],\n",
" [41.137],\n",
" [41.523],\n",
" [41.33 ],\n",
" [41.523],\n",
" [41.523],\n",
" [41.62 ],\n",
" [41.813],\n",
" [41.781],\n",
" [41.846],\n",
" [41.813],\n",
" [41.942],\n",
" [42.136],\n",
" [42.136],\n",
" [42.136],\n",
" [42.136],\n",
" [42.104],\n",
" [42.168],\n",
" [42.361],\n",
" [42.458],\n",
" [42.232],\n",
" [42.49 ],\n",
" [42.361],\n",
" [42.394],\n",
" [42.426],\n",
" [42.394],\n",
" [42.716],\n",
" [42.748],\n",
" [42.813],\n",
" [42.651],\n",
" [42.813],\n",
" [42.748],\n",
" [42.941],\n",
" [43.103],\n",
" [43.135],\n",
" [43.103],\n",
" [43.038],\n",
" [43.135],\n",
" [43.264],\n",
" [43.425],\n",
" [43.328],\n",
" [43.328],\n",
" [43.457],\n",
" [43.457],\n",
" [43.521],\n",
" [43.683],\n",
" [43.779],\n",
" [43.683],\n",
" [43.683],\n",
" [43.715],\n",
" [43.973],\n",
" [43.94 ],\n",
" [44.102],\n",
" [44.005],\n",
" [44.005],\n",
" [44.005],\n",
" [44.23 ],\n",
" [44.359],\n",
" [44.424],\n",
" [44.392],\n",
" [44.327],\n",
" [44.327],\n",
" [44.424],\n",
" [44.521],\n",
" [43.779],\n",
" [44.682],\n",
" [44.714],\n",
" [44.649],\n",
" [44.649],\n",
" [44.746],\n",
" [44.778],\n",
" [44.907],\n",
" [44.972],\n",
" [42.2 ],\n",
" [44.939],\n",
" [45.036],\n",
" [44.907],\n",
" [44.327],\n",
" [43.876],\n",
" [45.004],\n",
" [45.197],\n",
" [45.294],\n",
" [45.358],\n",
" [45.326],\n",
" [45.229],\n",
" [45.358],\n",
" [45.101],\n",
" [45.423],\n",
" [45.391],\n",
" [45.713],\n",
" [45.681],\n",
" [45.616],\n",
" [45.713],\n",
" [45.616],\n",
" [45.713],\n",
" [45.713],\n",
" [45.713],\n",
" [45.745],\n",
" [45.648],\n",
" [45.971],\n",
" [45.938],\n",
" [45.938],\n",
" [45.938],\n",
" [46.067],\n",
" [45.971],\n",
" [46.035],\n",
" [46.132],\n",
" [46.196],\n",
" [45.938],\n",
" [46.164],\n",
" [46.261],\n",
" [46.261],\n",
" [46.229],\n",
" [46.261],\n",
" [46.229],\n",
" [46.229],\n",
" [46.357],\n",
" [46.551],\n",
" [46.519],\n",
" [46.551],\n",
" [46.583]])"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"y = np.array(df['0']).reshape(300,1)\n",
"n = 300\n",
"t = np.linspace(0,n-1,n)\n",
"X = np.c_[np.ones(len(t)), t]\n",
"y"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 8.40650403],\n",
" [-13.57147156]])"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"np.random.seed(82)\n",
"eta = 0.00001 #lerning rate\n",
"n_iteration = 1000000\n",
"m = len(y)\n",
"theta = np.random.randn(2,1)*10\n",
"theta"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[25.5895366 ],\n",
" [ 0.07906986]])"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"for iterations in range(n_iteration):\n",
" gradient = 2/m * X.T.dot(X.dot(theta)- y)\n",
" theta = theta - eta*gradient\n",
"theta\n",
"#array([25.70275643, 0.07850281])\n",
"#array([[25.53711216],[ 0.07933242]]) -> 42\n",
"#array([[25.53941259],[ 0.0793209 ]]) -> 82"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# BGD Visualization"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"def plot_gradient_descent(eta):\n",
" m =len(y)\n",
" theta = np.random.randn(2,1)\n",
" plt.plot(t,y,'.b')\n",
" n_iteration = 1000000\n",
" Xnew1 = np.linspace(0,n-1,n)\n",
" Xnew = np.c_[np.ones(len(Xnew1)), Xnew1]\n",
" for iterations in range(n_iteration):\n",
" if iterations % 100000 == 0:\n",
" #print(iterations)\n",
" ypre = Xnew.dot(theta)\n",
" style = '-r' if iterations > 0 else 'g--'\n",
" plt.plot(Xnew1, ypre, style)\n",
" gradient = 2/m * Xnew.T.dot(Xnew.dot(theta)- y)\n",
" theta = theta - eta*gradient\n",
" plt.xlabel('$x_1$', fontsize=18)\n",
" #plt.axis([0,300, 15,50])\n",
" plt.title(r'$\\eta$ = {}'.format(eta), fontsize=16) "
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([[25.5895366 ],\n",
" [ 0.07906986]])"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAHVCAYAAADSAqClAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAACM5ElEQVR4nO3dd3iT1fsG8DspbYGWFsoqUMoeylSWdSCyERUFAcEBfgFBQBkyXYgLBGQ4QAUEHMhQpoO9lD0FRBDKaCsFZLSlha7k/P54fm9PQvdIk7T357p60bwZffMS7c05z3mOSSmlQEREROTCzM4+ASIiIqLMMLAQERGRy2NgISIiIpfHwEJEREQuj4GFiIiIXB4DCxEREbk8BhYiIiJyeQwsRERE5PIYWIiIiMjlMbAQERGRy2NgISIiIpfHwEJEDnHq1Cl8+umn6Nu3Lxo0aIAiRYrAZDLh/fffd+p5LV++HK1atUKpUqXg4+ODRo0aYcqUKUhKSsrweYmJifjkk0/w4IMPIiAgAEWLFkVQUBA6deqEpUuX5tPZExVeRZx9AkRUMM2ZMwezZs1y9mnYGT58OGbNmoUiRYqgdevW8PX1xZYtWzB27FisXbsWGzZsQLFixVI9LyIiAh06dMCJEydQpkwZPPDAA/Dx8UF4eDh27NgBHx8f9OzZ0wnviKjwYGAhIoeoX78+Ro0ahXvuuQf33nsvPvzwQ3z77bdOO59Vq1Zh1qxZ8PX1xfbt23HvvfcCAK5evYrWrVvjjz/+wFtvvYVp06bZPe/27dto164dTp48iXfeeQevv/46PD09U+6/desW/vnnn3x9L0SFEQMLETlE//797W6bzc6dgf7www8BAOPGjUsJKwBQpkwZzJ49Gw899BA+++wzvPXWW/D390+5f9KkSTh58iReeuklTJgwIdXrFi9eHI0bN3b4+RMVdqxhIXIDkyZNgslkwqhRo3D16lWMGjUKNWvWhLe3N8qXL4/+/fsjOjra2aeZp27fvo2PP/4Y9913H0qWLImiRYuiTp06GDNmDK5du5at1/r333+xf/9+AEDv3r1T3f/ggw+icuXKSEhIwK+//ppyPCkpCXPmzAEAjB49OhfvhohyiyMsRG7g4MGDAKTw8+6774avry8aNWqE4OBg7NixA/Pnz0dMTAyWLVvm5DPNGxcvXkTHjh1x7NgxBAQEoFmzZihRogQOHTqEqVOnYvny5di2bRuqVKmSpdc7fPgwACAgIADVqlVL8zFNmzZFeHg4Dh8+jF69egEADh06hKtXr6JixYqoWbMmjh07hhUrVuDixYsoVaoUHnroIXTq1Mnpo0dEhQH/KyNyA4cOHQIAzJ07F1OmTEFoaChWrlyJLVu2YN68eQCANWvWwGKxZOn1+vbtC5PJlO2vbdu2OeotplBKoUePHjh27Bj69euH8+fPY+PGjVixYgXOnDmD1157DefPn8eLL76Y5dc8d+4cACA4ODjdx1SuXNnusQBw9OhRAEBQUBDGjRuHRo0a4Z133sFXX32Fjz76CI899hiaNm2KsLCwnLxVIsoGjrAQubgbN26k/BKdNWsW+vbta3d/165d8eKLLyIhIQHJycnw8PDI9DUffPDBHJ1LYGBgjp6XHevXr8fOnTvRuHFjfPHFFyhSRP9vqkiRIpgyZQrWr1+PrVu34vjx46hfv36mr3nz5k0AgI+PT7qP8fX1BQDExMSkHDOmng4fPox9+/ZhyJAhePXVVxEYGJhy+/Dhw+jcuTMOHTpkV4xLRHmLgYXIxRmjKxUrVkxVyAroX8Zly5aFt7d3ll6zf//+ab6WK/jll18AAN26dbMLKwaz2YyWLVvi+PHj2LVrV5YCS04ppQBILUuvXr3w2WefpdzXtm1bbNy4EXXq1MHx48exZMkSPP/88w47F6LCjoGFyMUZgaVLly5p1kr89ddfAFBgVqqcPXsWAPDWW2/hrbfeyvCx//33HwCkFCPfaeHChQCAEiVKAADi4uLSfa3Y2FgAgJ+fX8ox43kAMHDgwFTPCQ4ORufOnfHTTz9h06ZNDCxEDsTAQuTijMDSvHnzNO83Vr+kd39a5s2bhz/++CPb5zJu3DjUrVs328/LDqvVCkCmrWrUqJHhY+vVqwcA+PHHH3HhwoVU9xuBpWrVqgCA8PDwdF/LuM94LABUr149ze9tGccjIyMzPFciyh0GFiIXZ6wQatq0aZr3G4GlWbNmWX7NP/74A4sWLcr2ufTt29fhgcUofu3SpQtGjRqVpeecP38+w/vvueceAFKTcu7cuTRXCh04cAAA7Hq03HvvvTCZTFBK4erVqynnZssY2TFqYIjIMbhKiMiF3bx5E2fOnEHx4sVx1113pfmYnASWhQsXQimV7a9WrVrlxdvKUKdOnQDInj9GDUluBQUFpVyfxYsXp7r/jz/+QHh4OLy9vfHoo4+mHA8MDEwpUN60aVOq5yUlJWH79u0AsjfCRUTZx8BC5MIOHz4MpRQaN26c5uqff//9FxcvXkTFihVRsWJFJ5xh3uvSpQuaNWuGffv24cUXX0ypU7F148YNfPHFF0hOTs7y677++usAgMmTJ6dMswEy6jJ48GAAwNChQ+263AJI6W47adIk7NmzJ+V4cnIyXnvtNZw9exYlSpTI1jJrIso+k8qrf8IQUZ6bMWMGRo4ciVdffTXNjQRXrVqFp556Cl26dMGqVavy/wQzcOjQoZQgAAChoaG4evUqgoKCUKlSpZTjK1euRIUKFeyee/HiRXTu3BlHjhxJ2VE5ODgYiYmJOHv2LI4dOwaLxYLbt2+jaNGiWT6nYcOG4ZNPPoGnpyfatGkDHx8fbN68GVFRUXjggQewcePGNDc/fP/99/HWW2+hSJEiaN68OQIDA3Ho0CGcP38exYoVw/Lly9G5c+ccXCUiyirWsBC5MGMkILP6FVecjoiJicHevXtTHY+IiEBERETK7YSEhFSPqVixIvbs2YOFCxdi6dKlOHr0KPbt24eAgABUrFgRgwYNwhNPPJGtsAJIH5sHHngAn3/+OXbt2oWkpCTUqFED48aNw4gRI+Dl5ZXm89588000b94cM2fOxN69e7F//34EBgaib9++GDt2rMPreoiIIyxEbq19+/bYuHEjNmzYgHbt2jn7dIiIHIaBhciNBQQE4MaNG7h+/TpKlSrl7NMhInIYFt0SuakzZ87gxo0bqFmzJsMKERV4DCxEbiony5mJiNxVtgLLO++8k2r3Vttis/j4eAwZMgSlS5eGr68vunXrhsuXL+f5SRMR0KtXLyil0uwrQkRU0GR7hKVevXqIjIxM+bJt7z1ixAisXbsWy5cvx/bt23Hx4kV07do1T0+YiIiICp9sL2suUqRImlvMR0dHY/78+Vi8eDFat24NAFiwYAHuuusu7NmzB/fdd1/uz5aIiIgKpWwHltOnT6NixYooWrQoQkJCMGnSJAQHB+PgwYNISkpC27ZtUx5bt25dBAcHY/fu3ekGloSEBLs+DFarFdevX0fp0qVhMply8JaIiIgovymlcPPmTVSsWDHNneVzK1uBpUWLFli4cCHq1KmDyMhITJw4EQ899BCOHz+OS5cuwcvLCyVLlrR7Tvny5XHp0qV0X3PSpEmYOHFijk6eiIiIXEt4eDiCgoLy/HWzFViMTckAoGHDhmjRogWqVKmCZcuWpdnOOivGjx+PkSNHptyOjo5GcHAwwsPD4efnl6PXJCKiQuzTT4E33wRMJsBoNeblBSQmAiVKADdv6j+N4wBgNgNWK1C0KBAfDxQvDty6Bfj7A6tWATY7eVNqMTExqFy5MkqUKOGQ189Va/6SJUuidu3aOHPmDNq1a4fExERERUXZjbJcvnw5zZoXg7e3N7y9vVMd9/PzY2AhIqLs2bABePtt+d7LC0hIAHx9gdhYwM8PiIkBfHwkrBQposNKkSJAcjJQrBhw+7YOK6VKARs3Ak2aOO89uRlHlXPkapIpNjYWoaGhqFChApo0aQJPT09s3rw55f5Tp04hLCwMISEhuT5RIiKiDIWGAs88I6Mkvr4SVkqUkLDi4yNhxdsbiIuT0Rdjt29PT/uwUqy
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"np.random.seed(112)\n",
"plot_gradient_descent(eta=0.000001)\n",
"theta"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[<matplotlib.lines.Line2D at 0x1443cc2f0>]"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAABOL0lEQVR4nO3de5zMdf//8cfMspvTLtZhaVliUcRVLrHVJaEk9VN0oLo66aCQU8ihRGrF5tRVLmrTdXUlqW+HS8dLRFc5JBElmxUxsZSy65DF7uf3x/uatbPHmdk57czzfrvtzX4+M/OZ935azdP78HrbLMuyEBEREQkQe7AbICIiIpFF4UNEREQCSuFDREREAkrhQ0RERAJK4UNEREQCSuFDREREAkrhQ0RERAJK4UNEREQCqkqwG1BUfn4++/bto1atWthstmA3R0RERNxgWRZHjhyhcePG2O1l922EXPjYt28fTZo0CXYzRERExAt79+4lMTGxzOeEXPioVasWYBofGxsb5NaIiIiIO3JycmjSpEnB53hZQi58OIdaYmNjFT5EREQqGXemTGjCqYiIiASUR+Hj8ccfx2azuXy1adOm4PETJ04wZMgQ4uPjqVmzJv379+fAgQM+b7SIiIhUXh73fLRt25b9+/cXfH3++ecFj40cOZJly5bxxhtvsHr1avbt20e/fv182mARERGp3Dye81GlShUSEhKKnc/OziY9PZ3FixfTvXt3ABYtWsS5557LunXr6NKlS8VbKyIiIpWexz0fO3bsoHHjxpxzzjnceuut7NmzB4CNGzdy6tQpevbsWfDcNm3a0LRpU9auXVvq9XJzc8nJyXH5EhERkfDlUfjo3LkzL7/8Mh999BHz589n165d/OUvf+HIkSNkZWURHR1N7dq1XV7TsGFDsrKySr1mamoqcXFxBV+q8SEiIhLePBp26d27d8H37du3p3PnziQlJbF06VKqVavmVQPGjx/PqFGjCo6d64RFREQkPFVoqW3t2rVp1aoVmZmZJCQkcPLkSQ4fPuzynAMHDpQ4R8QpJiamoKaHanuIiIiEvwqFj6NHj7Jz504aNWpEx44dqVq1KitWrCh4PCMjgz179pCSklLhhoqIiEh48GjY5eGHH+baa68lKSmJffv2MXnyZKKiohg4cCBxcXEMGjSIUaNGUbduXWJjYxk2bBgpKSla6SIiIiIFPAofDoeDgQMHcujQIerXr8+ll17KunXrqF+/PgCzZ8/GbrfTv39/cnNz6dWrF88//7xfGi4iIiKVk82yLCvYjSgsJyeHuLg4srOzNf9DRESkkvDk81t7u4iIiESK/fvhjjvgnXeC2gyFDxERkXCXmwszZkCrVvDPf8LDD8Pp00Frjsfl1UVERKQS+eADGDECduwwx507w7x5UCV4EUA9HyIiIuFoxw645hro08d837AhvPwyrFkDF10U1KYpfIiIiISTI0dg3Dho2xbefx+qVjXDLD/8YOZ72IP/0a9hFxERkXCQnw+vvgpjx4JzT7WrroI5c6B166A2rSiFDxERkcruq6/goYfAuYt8ixYmdPTpAzZbUJtWkuD3vYiIiIh3Dh6Ee+4xczjWroUaNSA1Fb77zsz3CMHgAer5EBERqXxOnYLnnoPHH4fsbHPutttg+nQ4++ygNs0dCh8iIiKVyfLlMHw4fP+9Ob7wQrN09pJLgtsuD2jYRUREpDL48Ue4/nq48koTPOrVg4UL4csvK1XwAPV8iIiIhLZjx8xwysyZplJpVBQMHQqTJ0OdOsFunVcUPkREREKRZcHSpaZGh8NhzvXoAXPnmhoelZjCh4iISKj55huzdPazz8xxs2bwzDNm2CVEV7B4QnM+REREQsWhQ/Dgg2YS6WefQbVqMHUqbNsG/fqFRfAA9XyIiIgE3+nTZvLopEnw++/m3E03mXkeTZsGt21+oPAhIiISTKtWmSGWrVvN8fnnm6Wz3boFs1V+pWEXERGRYNizB26+GS6/3ASPOnVM4bCvvw7r4AHq+RAREQmsP/6AtDRTBv2PP8wus/ffD088AfHxwW5dQCh8iIiIBIJlwdtvw+jRsHu3OfeXv5ghlj/9KZgtCziFDxEREX/77jtTEn3FCnOcmGh6P266KWxWsHhCcz5ERET85fBhGDECOnQwwSMmxqxo2b7dzPeIwOAB6vkQERHxvbw8eOklmDABfv3VnLvuOlMo7Jxzgtq0UKDwISIi4ktffGGWzn79tTk+91xTEv2KK4LbrhCiYRcRERFf2LcP/vpXuPRSEzxiY2H2bFMqXcHDhXo+REREKiI314SMadPMDrQ2G9x9Nzz1FDRoEOzWhSSFDxEREW9YFrz/vplQunOnOdelCzz7LPz5z0FtWqjTsIuIiIinMjKgTx+49loTPBIS4J//NPM9FDzKpZ4PERERd+XkmEqkc+aYzeCqVoWRI83y2Vq1gt26YhwO2LEDkpPN8Y4dULMmHD1qziUmBqddCh8iIiLlyc+HV16BcePgwAFzrk8fmDULWrUKbttKkZ4O991nmg5mKoplnXncbjcb6Q4aFPi2adhFRESkLBs2wMUXw513muCRnAzvvWe+QjR4OByuwQNcgweYx+6/3zw30BQ+RERESnLggFm1ctFFsH69Ga94+mmzA22fPsFuXZl27HANHqXJy4PMTP+3pygNu4iIiBR28iT87W8wZYqZ4wFw++0wfTo0ahTw5hSet+Gco+FwwJo15vvmzWHXLtfv//1v964dFQUtW/q+zeVR+BAREXH6z3/MBnDbt5vjjh3N0tmUlKA0Jy3NTDPJzzdzNKZPh4MHTZX2osMonoqKggULgjPpVOFDRERk504YNepMl0H9+pCaCnfdZT71A6DoypRp00w4cMrPh7FjK/Yedjs88ogpuNqypVa7iIiIBN7RoyZkpKWZ4ZYqVWDYMHjsMahd2y9v6QwZNWueGS7ZssU0w515Gt6w22HJEtOBE6zAUZjCh4iIRB7LgtdeM10JP/9szvXsaTaAO+88v71t0eWvgeAcXrnxxsC9Z3kUPkREJLJs2mR2nf38c3PcrJnZm6VvX1MMww+cE0QDGTxsNhg92kxhCYXejsIUPkREJDL8+qupRLpwoen5qF4dxo83n9DVqnl0qcKrTS6+2PxZUiXRXbtg5Up44QXfh45rrjGLcJo1g927zbnC34fKEEtJbJZV0fmyvpWTk0NcXBzZ2dnExsYGuzkiIlLZnT4Nf/87PPooHD5szg0YADNmQJMmHl8uPR3uvdd1tUnh6qFFK4lW1G23weLFruHFboeffgqtcOHJ57eKjImISPhauRIuuMBMIj18GDp0gNWrzXwPL4KHw1E8eIDrcUWCh812ZuTHbjf56JVXTGdNVJQ5HxVljkMpeHhKwy4iIhJ+fvoJHn4Y3nzTHNetC08+aZKD81PcQw7HmREbX5g/H+LjzffNmsGxY2cKfmVmui6FHTQIevUqfr6yUvgQEZHwcfy46S54+mk4ccJ0HzzwAEydagJIKYpWDHXu+gpm/sbGjWZhjK+CR1SUmbNRWogo6XxiYuUPHU4KHyIiUvlZFvzf/5nJo3v2mHOXXQbz5kH79mW+tKQ5HE6+nL/hvFYwK4uGCoUPERGp3LZuNetJP/3UHDdpYoqG3XhjuUtnN2woPXhA+cHj2mvhgw/MBm3Ot7KsM5VE//Qnc85ZnT1chk0qqkITTqdPn47NZmPEiBEF57p164bNZnP5Gjx4cEXbKSIi4ur33029jgsuMMEjJsZUJt2+HW66qSANOBzm4aJbx6enQ+fO3vdsREXB88+bpa2ffmo6XPbsMd//9JOZYnLjjebLOWTSrZuCB1Sg52PDhg0sWLCA9iV0Z917771MnTq14Lh69erevo2IiIirvDx48UWYOBEOHTLn+vWDtDQcVZuzY/2Z+Rpz58KsWWc2Zlu40EzedDhMwS9vg4fd7jp0UjhQKFyUz6vwcfToUW699VZeeOEFpk2bVuzx6tWrk5CQUOHGiYiIuPj8c7NsdvNmc3zeeWZeR48eLjvAljRXIz/fBI5evUwo8abol81mejvKmiwq5fNq2GXIkCH
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.plot(t,y,'.b')\n",
"Xnew1 = np.linspace(0,n-1,n)\n",
"Xnew = np.c_[np.ones(len(Xnew1)), Xnew1]\n",
"ypre = Xnew.dot(theta)\n",
"plt.plot(Xnew1, ypre, '-r')"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA04AAAGXCAYAAACEDgBPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/H5lhTAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB99ElEQVR4nO3dd3zM9x8H8Nf3Mi6JbAlBgtgrNhFFac0mJLYoRc1WdaFoq+jSotVWiyqli8ROJGqrGStij9o7iJGERNZ9fn/cL0cqZF4+9728no/HPeruvnf3uqvv2/f9HZ+PIoQQICIiIiIiomfSyA5ARERERERk6tg4ERERERER5YCNExERERERUQ7YOBEREREREeWAjRMREREREVEO2DgRERERERHlgI0TERERERFRDtg4ERERERER5YCNExERERERUQ7YOBEREREREeWAjRMREREREVEO2DiRyTl9+jRmzZqFgQMHwsfHB5aWllAUBZ9//rnUXMuWLUPr1q3h4uKCEiVKoF69epg2bRrS0tKe+7rU1FT88MMPaNGiBVxdXWFjYwNPT0906tQJoaGhRZSeqHjK73prrPfN6+tMtR4SmTtTXffyU3suX76Mt956C9WrV4etrS1sbGzg7e2NAQMG4PDhw0WY3gwIIhPzzjvvCABP3T777DPpmSwtLUX79u1Ft27dhLOzswAgWrRoIZKSkrJ93ZUrV0StWrUEAOHm5iYCAgJE7969RfPmzYWdnZ3o3r17EX8TouIjv+utsd43P68zxXpIVByY4rqXnxqyZ88e4eDgIACIcuXKiS5duoiuXbsKb29vw3stXbpUwrdRJzZOZHJ++eUXMWbMGPHXX3+JkydPiv79+0stVqtWrRIAhL29vYiOjjY8fvv2beHj4yMAiNGjRz/1uqSkJFGjRg0BQEyePFmkpqZmef7hw4ciJibG2PGJiqX8rrfGet/8vs7U6iFRcWFq615+a0jdunUFADFs2LAs2yEZGRni448/FgCEs7OzSE5OLpLvoXZsnMjkDRgwQGqxatKkiQAgPv/886ee27FjhwAgtFqtuH//fpbnJk6caChWRFS08rveGut9CyuP7HpIVFzJXvfyU0Pi4uIMR8pu3br11OvS09OFra2tACAOHjxo1Pzmgtc4mbmpU6dCURSMGTMGcXFxGDNmDKpUqQKtVovSpUtjyJAhiI+Plx2zUCUnJ+Obb75Bs2bN4OzsDBsbG1SvXh0ffPAB7ty5k6f3unbtGvbv3w8A6Nu371PPt2jRAl5eXkhJScHatWsNj6elpWHOnDkAgLFjxxbg2xDJp7Y6kt/11ljva6w8RGqhthpSGExhW0Sr1eb6M9zc3PKUqbhi42TmoqOjAegHKKhVqxZWrlwJHx8fvPDCC7hz5w4WLFiAoUOHSk5ZeK5fvw5fX1+MGTMGZ86cQZMmTfDKK68gJSUF06dPR+PGjXHp0qVcv19MTAwAwNXVFd7e3tku07hx4yzLAsDBgwcRFxeHsmXLokqVKjh69CimTJmC4cOHY/z48YiMjIROpyvANyUqOmqrI/ldb431vsbKQ6QWaqshBWUq2yL29vZo2bIlAODjjz/OMoCETqfD5MmTkZycjE6dOsHLyyvP37M4YuNk5g4ePAgA+OWXXzBt2jScO3cOq1atwpYtWzB//nwAQHh4ODIyMnL1fgMHDoSiKHm+/fPPP8b6igZCCPTq1QtHjx7F4MGDcfHiRWzcuBErV67E2bNnMXr0aFy8eBGDBg3K9XteuHABAFC+fPlnLpNZbDKXBYAjR44AADw9PTF+/HjUq1cPkydPxrx58/D1118jICAAjRs3xuXLl/PzVYmKlNrqSH7XW2O9r7HyEKmF2mpIQZjStgig/80rVaqEefPmwdvbG0FBQejevTuqVKmCadOmoX///ggJCcnHNy2eLGUHIOO5d++eYQX6/vvvMXDgwCzPd+vWDYMGDUJKSgrS09NhYWGR43u2aNEiX1k8PDzy9bq8WL9+PXbt2oX69etj7ty5sLR8/Nfb0tIS06ZNw/r167F161YcO3YMderUyfE9ExMTAQAlSpR45jL29vYAgISEBMNjmYfhY2JisG/fPowcORJvv/02PDw8DPdjYmLg7++PgwcPwsrKKl/fmcjY1FhH8rveGut9jZWHSA3UWEMKwpS2RQCgevXqiIqKQv/+/bFhwwZcu3bN8FytWrXQunVrODo65uk7FmdsnMxY5h6esmXLYsiQIU89n7kiuru75/o82CFDhmT7XqYgMjISANC9e/cshSqTRqNBq1atcOzYMezevTtXxSq/hBAA9Nc6BQcH48cffzQ817ZtW2zcuBHVq1fHsWPHEBISgv79+xstC1FBFLc6QkSFq7jVEFPaFgGAXbt2oVu3brC0tMTixYvx0ksvwdraGrt27cL777+PwYMHY9euXViwYIFRc5gLNk5mLLNYBQYGQqN5+qzM48ePAwDq169flLGM5vz58wCAiRMnYuLEic9d9vbt2wBguFD1vxYtWgQAcHBwAAA8fPjwme/14MEDAMiyxybzdQAwfPjwp15Tvnx5+Pv7Y8WKFdi0aRMbJzJZaqwj+V1vjfW+xspDpAZqrCEFYUrbIvfv30fXrl0RFxeHqKgo+Pr6Gp4LCAhArVq14OPjg19//RX9+vVDmzZtcvENizc2TmYss1g1bdo02+czR2h51vPZmT9/Pnbu3JnnLOPHj0eNGjXy/Lq8yBxsoUWLFqhcufJzl61duzYAYPny5dleoJlZrCpWrAgAuHLlyjPfK/O5zGUBoFKlStn++UmZj9+4ceO5WYlkUmMdye96a6z3NVYeIjVQYw0pCFPaFomMjMTt27dRuXLlLE1TpkqVKsHX1xdbt27Fpk2b2DjlAhsnM5Y5ik3mSCv/lVmsmjRpkuv33LlzJ3777bc8Zxk4cKDRi1XmhZGBgYEYM2ZMrl5z8eLF5z7foEEDAPprli5cuJDtaDYHDhwAADRs2NDwWMOGDaEoCoQQiIuLy3a0msy9S5nnJROZIjXWkfyut8Z6X2PlIVIDNdaQgjClbZHMAaiedyTbyckJAHD37t1cZS3uOKqemUpMTMTZs2dhZ2eHmjVrZrtMforVokWLIPQTJ+fp1rp168L4Ws/VqVMnAMCyZcsM1xgVlKenp+H3Wbx48VPP79y5E1euXIFWq8Urr7xieNzDw8Nw8eqmTZueel1aWhq2bdsGIG972YiKklrrSH7XW2O9r7HyEJk6tdaQgjClbZFy5coBAE6dOpXtPFlpaWmGI4LPGuac/sMo0+qSdNu2bRMARPPmzbN9/urVqwKAKFu2bBEny7vcztadkZFhmFl7wIAB2c6SfffuXTFnzhyRlpaW689ftWqVACDs7e1FdHS04fG4uDjh4+MjAIjRo0c/9bpNmzYJAMLFxUVERUUZHk9LSxOjRo0SAISDg4OIjY3NdRaioqTmOpLf9VYIIcaPHy+qV68uxo8fX2jvW5A8T8ptPSQyBWquIf+lxm2RW7duiRIlSggAomfPniIxMdHwXEpKihg5cqQAIKysrMS5c+dynaU4Y+Nkpr799lsBQLz99tvZPp+5AgYGBhZtsFyIjo4Wvr6+hpubm5sAIDw9PbM8fv369adee+3aNVG/fn0BQJQoUUI0b95c9OnTR3Tr1k3Ur19fWFhYCAAiOTk5T5nefvttQ3Hp2LGj6N69u3B2dhYAxAsvvCCSkpKyfd1nn30mAAhLS0vRvHlz0a1bN1GxYkUBQNja2oqIiIh8/UZERUHNdUSI/K+3mRtIAwYMKNT3zc/rClIPiWRTcw0xl22RP/74Q1haWgoAwt3dXbzyyisiMDBQlCtXTgAQGo1GzJkzJ9+/U3HDxslM9evXTwAQv//+e7bPf/jhhwKA+OKLL4o4Wc62bt0qAOR4u3DhQravf/TokZg7d65o06aNKFmypLC0tBSlSpUS9evXFyNHjhTr16/PV67Q0FDRqlUr4ejoKGxtbUWdOnXEV199JVJSUp77uvXr14tOnToJV1dXYWVlJby8vMTAgQPFyZMn85WDqKiouY5kys96m1PjlN/3zc/rClo
"text/plain": [
"<Figure size 1000x400 with 3 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.figure(figsize=(10,4))\n",
"plt.subplot(131)\n",
"np.random.seed(112)\n",
"plot_gradient_descent(eta=0.000001)\n",
"plt.subplot(132)\n",
"np.random.seed(112)\n",
"plot_gradient_descent(eta=0.001)\n",
"plt.subplot(133)\n",
"np.random.seed(112)\n",
"plot_gradient_descent(eta=0.00000001)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}