diff --git a/4-ann/main.ipynb b/4-ann/main.ipynb new file mode 100644 index 0000000..61ca523 --- /dev/null +++ b/4-ann/main.ipynb @@ -0,0 +1,1127 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "d11e7334", + "metadata": {}, + "source": [ + "# ANN - Example 3" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "2b74c408", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "W = np.array([[0.9, 0.3, 0.4],\n", + " [0.2, 0.8, 0.2],\n", + " [0.1, 0.5, 0.6]])\n", + "I = np.array([[0.9],[0.1], [0.8]])\n", + "\n", + "def mySigmoid(x):\n", + " return 1/(1+np.exp(-x))" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "a24d741f", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[1.16],\n", + " [0.42],\n", + " [0.62]])" + ] + }, + "execution_count": 8, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Xih = np.dot(W,I)\n", + "Xih" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "e977d118", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[0.76133271],\n", + " [0.60348325],\n", + " [0.65021855]])" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "Oh = mySigmoid(Xih)\n", + "Oh" + ] + }, + { + "cell_type": "markdown", + "id": "9801c7d7", + "metadata": {}, + "source": [ + "# Creating Class and Methods\n", + "A class defines the structure, data and methods that an object will have. There is possible to have public and private variables to operate in the methods.\n", + "\n", + "\n", + "- Argument\n", + "- Initialitation\n", + "- Methods\n", + "- Destroy" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "ec1cead5", + "metadata": {}, + "outputs": [], + "source": [ + "class Dog:\n", + " # init method\n", + " def __init__(self, dogName, dogAge):\n", + " self.name=dogName\n", + " self.age=dogAge\n", + " \n", + " # status method\n", + " def status(self):\n", + " print(\"The Dogs name is: \", self.name)\n", + " print(\"The Dogs age is: \", self.age)\n" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "5a484af0", + "metadata": {}, + "outputs": [], + "source": [ + "perro1=Dog('cuadrado', 9)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "62857352", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The Dogs name is: cuadrado\n", + "The Dogs age is: 9\n" + ] + } + ], + "source": [ + "perro1.status()" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "7e848bc3", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "cuadrado\n", + "9\n" + ] + } + ], + "source": [ + "print(perro1.name)\n", + "print(perro1.age)" + ] + }, + { + "cell_type": "markdown", + "id": "9fb56224", + "metadata": {}, + "source": [ + "# The Neural Notwork Class\n", + "\n", + "The next chunk of code defines the Neural Network's basic structure. We are going to implement and define the methods one at tima to understand them in a better way.\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0a8eb25f", + "metadata": {}, + "outputs": [], + "source": [ + "class NeuralNetwork:\n", + " # init method\n", + " def __init__():\n", + " pass\n", + " \n", + " # NN computing method\n", + " def feedforward():\n", + " pass\n", + " \n", + " # NN trainning method \n", + " def backpropagation():\n", + " pass" + ] + }, + { + "cell_type": "markdown", + "id": "9253fc86", + "metadata": {}, + "source": [ + "## Initialization or creation Method\n", + "\n", + "Let’s begin with the initialization. We know we need to set the number of input, hidden and output layer nodes. That defines the shape and size of the neural network. Thus, we’ll let them be set when a new neural network object is created by using the class' parameters. That way we retain the choice to create new neural networks of different sizes with simple methods.\n", + "\n", + "A good programmers, computer scientists and mathematicians, try to create more general code rather than specific code. It is a good habit, because it forces us to think about solving problems in a deeper and more general way. This means that our code can be used in more general scenarios.\n", + "\n", + "Then, ket us see how our code should look like:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "b4431606", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "\n", + "class NeuralNetwork:\n", + " # init method\n", + " def __init__(self, inputN,hiddenN, outputN, lr):\n", + " # creates a NN with three layers (input, hidden, output)\n", + " # inputN - Number of input nodes\n", + " # hiddenN - Number of hidden nodes\n", + " self.inN=inputN\n", + " self.hiN=hiddenN\n", + " self.outN=outputN\n", + " self.lr=lr\n", + " \n", + " #weight W11 W21 W31\n", + " # W12 W22 W32\n", + " # .....\n", + " self.wih=np.random.rand(self.hiN, self.inN)\n", + " self.who=np.random.rand(self.outN,self.hiN)\n", + " pass\n", + " \n", + " # NN computing method\n", + " def feedforward():\n", + " pass\n", + " \n", + " # NN trainning method \n", + " def backpropagation():\n", + " pass" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "d6eb9df0", + "metadata": {}, + "outputs": [], + "source": [ + "myNN=NeuralNetwork(3,3,3,0.1)" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "id": "b874c4df", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[0.86760733, 0.61494357, 0.16001254],\n", + " [0.86192173, 0.71746821, 0.90211204],\n", + " [0.44345426, 0.66751851, 0.33738905]])" + ] + }, + "execution_count": 18, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "myNN.wih" + ] + }, + { + "cell_type": "markdown", + "id": "4975a68b", + "metadata": {}, + "source": [ + "At this point were are only creating an object, but the myNN instance can't do any useful yet. Also, this is a good technique to start coding somethig, by keeping it small at the begining (make commits), and then grow the methods.\n", + "\n", + "Next, we should add more code to allow our NN class finish its initialization by creating the weight matrixes." + ] + }, + { + "cell_type": "markdown", + "id": "544d454f", + "metadata": {}, + "source": [ + "## Feedfordward method and weights initialization\n", + "\n", + "So the next step is to create the network of nodes and links. The most important part of the network is the link weights. They’re used to calculate the signal being fed forward, the error as it’s propagated backwards, and it is the link weights themselves that are refined in an attempt to to improve the network.\n", + "\n", + "For the basic NN, the weight matrix consist of:\n", + "\n", + "- A matrix that links the input and hidden layers, $Wih$, of size hidden nodes by input nodes ($hn×in$)\n", + "- and another matrix for the links between the hidden and output layers, $Who$, of size $on×hn$ (output nodes by hidden nodes)\n", + "\n", + "$$X_h=W_{ih}I$$\n", + "$$O_h=\\sigma{X_h}$$" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "id": "ce35cb31", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "\n", + "\n", + "class NeuralNetwork:\n", + " # init method\n", + " def __init__(self, inputN,hiddenN, outputN, lr):\n", + " # creates a NN with three layers (input, hidden, output)\n", + " # inputN - Number of input nodes\n", + " # hiddenN - Number of hidden nodes\n", + " self.inN=inputN\n", + " self.hiN=hiddenN\n", + " self.outN=outputN\n", + " self.lr=lr\n", + " \n", + " #weight W11 W21 W31\n", + " # W12 W22 W32\n", + " # .....\n", + " np.random.seed(42) \n", + " self.wih=np.random.rand(self.hiN, self.inN)\n", + " self.who=np.random.rand(self.outN,self.hiN)\n", + " pass\n", + " \n", + " # NN computing method\n", + " def feedforward(self, inputList):\n", + " # computing hidden output\n", + " inputs = np.array(inputList, ndmin=2).T\n", + " self.Xh = np.dot(self.wih, inputs)\n", + " self.af = lambda x:1/(1+np.exp(-x))\n", + " self.Oh = self.af(self.Xh)\n", + " \n", + " # computing output \n", + " self.Xo = np.dot(self.who, self.Oh)\n", + " self.Oo = self.af(self.Xo)\n", + " return self.Oo\n", + " \n", + " # NN trainning method \n", + " def backpropagation():\n", + " pass" + ] + }, + { + "cell_type": "code", + "execution_count": 88, + "id": "c267a481", + "metadata": {}, + "outputs": [], + "source": [ + "myNN=NeuralNetwork(3,5,3,0.3)" + ] + }, + { + "cell_type": "markdown", + "id": "6c07d6dc", + "metadata": {}, + "source": [ + "At this point we can review the variables or class attributes by calling them:" + ] + }, + { + "cell_type": "code", + "execution_count": 89, + "id": "eff1bde8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[0.37454012, 0.95071431, 0.73199394],\n", + " [0.59865848, 0.15601864, 0.15599452],\n", + " [0.05808361, 0.86617615, 0.60111501],\n", + " [0.70807258, 0.02058449, 0.96990985],\n", + " [0.83244264, 0.21233911, 0.18182497]])" + ] + }, + "execution_count": 89, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "myNN.wih" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "id": "159d06d2", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[0.72922423],\n", + " [0.74638741],\n", + " [0.77466214]])" + ] + }, + "execution_count": 90, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "myNN.feedforward([0.3, 0.2, 0.1])" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "id": "379439b8", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[0.59283661],\n", + " [0.55635965],\n", + " [0.56236646],\n", + " [0.57774658],\n", + " [0.57697877]])" + ] + }, + "execution_count": 91, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "myNN.Oh" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "id": "4c351ffd", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[0.37570429],\n", + " [0.22640073],\n", + " [0.25077181],\n", + " [0.31352966],\n", + " [0.31038311]])" + ] + }, + "execution_count": 92, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "myNN.Xh" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "id": "e95a288d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([0.72644667, 0.6712605 , 0.63947874])" + ] + }, + "execution_count": 77, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "myNN.Oo" + ] + }, + { + "cell_type": "markdown", + "id": "15769565", + "metadata": {}, + "source": [ + "## The backpropagation and trainning method\n", + "\n", + "$$ \\frac{\\partial E}{\\partial w_{jk}}= -e_j\\cdot \\sigma\\left(\\sum_i w_{ij} o_i\\right) \\left(1-\\sigma\\left(\\sum_i w_{ij} o_i\\right) \\right) o_i $$" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "3006bfec", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "\n", + "\n", + "class NeuralNetwork:\n", + " # init method\n", + " def __init__(self, inputN,hiddenN, outputN, lr):\n", + " # creates a NN with three layers (input, hidden, output)\n", + " # inputN - Number of input nodes\n", + " # hiddenN - Number of hidden nodes\n", + " self.inN=inputN\n", + " self.hiN=hiddenN\n", + " self.outN=outputN\n", + " self.lr=lr\n", + " \n", + " #weight W11 W21 W31\n", + " # W12 W22 W32\n", + " # .....\n", + " np.random.seed(40) \n", + " self.wih=np.random.rand(self.hiN, self.inN)-0.5\n", + " self.who=np.random.rand(self.outN,self.hiN)-0.5\n", + " pass\n", + " \n", + " # NN computing method\n", + " def feedforward(self, inputList):\n", + " # computing hidden output\n", + " inputs = np.array(inputList, ndmin=2).T\n", + " self.Xh = np.dot(self.wih, inputs)\n", + " self.af = lambda x:1/(1+np.exp(-x))\n", + " self.Oh = self.af(self.Xh)\n", + " \n", + " # computing output \n", + " self.Xo = np.dot(self.who, self.Oh)\n", + " self.Oo = self.af(self.Xo)\n", + " return self.Oo\n", + " \n", + " # NN trainning method \n", + " def backpropagation(self, inputList, targetList):\n", + " # data\n", + " lr = self.lr \n", + " inputs = np.array(inputList, ndmin=2).T\n", + " target = np.array(targetList, ndmin=2).T\n", + " \n", + " #computting hidden layer\n", + " Xh = np.dot(self.wih, inputs)\n", + " af = lambda x:1/(1+np.exp(-x))\n", + " Oh = af(Xh)\n", + " \n", + " # computing output \n", + " Xo = np.dot(self.who, Oh)\n", + " Oo = af(Xo)\n", + " \n", + " # Output error\n", + " oe = target-Oo\n", + " # E propagation\n", + " hiddenE = np.dot(self.who.T, oe)\n", + " \n", + " # updating weights\n", + " self.who+=lr*np.dot(oe*Oo*(1-Oo), Oh.T) \n", + " self.wih+=lr*np.dot(hiddenE*Oh*(1-Oh), inputs.T) \n", + " return self.wih, self.who" + ] + }, + { + "cell_type": "markdown", + "id": "05883c85", + "metadata": {}, + "source": [ + "## Exam" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "a97b0ca8", + "metadata": {}, + "outputs": [], + "source": [ + "NN3 = NeuralNetwork(3,3,3,0.15)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "0a101372", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[-0.09231297, -0.44463396, 0.28853488],\n", + " [-0.21269482, -0.04964941, -0.19608769],\n", + " [ 0.02639952, 0.12381221, 0.27677546]])" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "NN3.wih" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "8c4ed22a", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[ 0.18624165, 0.48093886, 0.10081609],\n", + " [ 0.31396852, 0.20864515, -0.47246532],\n", + " [ 0.40426722, -0.05009515, -0.38107535]])" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "NN3.who" + ] + }, + { + "cell_type": "markdown", + "id": "1b1d8aa2", + "metadata": {}, + "source": [ + "## First Feed" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "a74e4922", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[0.58611654],\n", + " [0.48786288],\n", + " [0.4846667 ]])" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "NN3.feedforward([0.43, 0.88, 0.95])" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "d55ae872", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(array([[-0.09474099, -0.44960292, 0.28317066],\n", + " [-0.21423683, -0.05280516, -0.19949447],\n", + " [ 0.02659345, 0.12420909, 0.2772039 ]]),\n", + " array([[ 0.18060506, 0.47579807, 0.09354318],\n", + " [ 0.3176326 , 0.21198694, -0.46773754],\n", + " [ 0.3976255 , -0.05615266, -0.38964518]]))" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "NN3.backpropagation([0.43, 0.88, 0.95], [0.25, 0.7, 0.1])" + ] + }, + { + "cell_type": "markdown", + "id": "86153af2", + "metadata": {}, + "source": [ + "## Second feed" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "id": "6b7af517", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[0.4262625 ],\n", + " [0.50988177],\n", + " [0.52984418]])" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "NN3.feedforward([0.47, 0.07, 0.64])" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "id": "388f333c", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "(array([[ 0.34418188, 0.05427654, -0.04100328],\n", + " [-0.14923924, 0.08653238, 0.03279852],\n", + " [ 0.32666948, -0.23622231, -0.02194774],\n", + " [-0.15480446, -0.36651948, -0.24284465]]),\n", + " array([[-0.44498126, -0.09860092, -0.12175549, 0.0634736 ],\n", + " [ 0.39189539, -0.33801846, -0.09787605, 0.14729404],\n", + " [ 0.04218888, 0.0429173 , -0.00094672, 0.08642189]]))" + ] + }, + "execution_count": 47, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "NN3.backpropagation([0.47, 0.07, 0.64], [0.17, 0.67, 0.14])" + ] + }, + { + "cell_type": "markdown", + "id": "c146df50", + "metadata": {}, + "source": [ + "## Third feed" + ] + }, + { + "cell_type": "markdown", + "id": "a2510fdf", + "metadata": {}, + "source": [ + "## Fourth feed" + ] + }, + { + "cell_type": "markdown", + "id": "2599e87a", + "metadata": {}, + "source": [ + "## Fifth feed" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8afe16a9", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b46b2eff", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6401ecc0", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "9b35dffb", + "metadata": {}, + "source": [ + "# MNIST Dataset\n", + "\n", + "Reading the complete file" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "b430564a", + "metadata": {}, + "outputs": [], + "source": [ + "data_file= open(\"mnist_train.csv\", 'r')\n", + "data_list= data_file.readlines()\n", + "data_file.close()" + ] + }, + { + "cell_type": "markdown", + "id": "dcacef2d", + "metadata": {}, + "source": [ + "## Interpreting one intrance" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "b7e043f7", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAam0lEQVR4nO3dfUyV9/3/8dfxhuMdHIcIByY61FZdVcycMmL1awdDWea8+0Nrl2hnNDpsqs61cWu1uiVsNukaDdMs2bRN6s1MqqZmM7NYMN3QRapxpi0Tw6oOwdUNDmJFI5/fH8az31GsXngObw4+H8mVyDnXh+vda1d47vIcDz7nnBMAAB2sm/UAAIDHEwECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmelgPcLfW1lbV1tYqMTFRPp/PehwAgEfOOTU1NSkjI0Pdut3/PqfTBai2tlaZmZnWYwAAHtGFCxc0aNCg+z7f6QKUmJgo6fbgSUlJxtMAALwKhULKzMwM/zy/n5gFqKSkRK+//rrq6uqUnZ2tLVu2aOLEiQ9cd+ev3ZKSkggQAMSxB72MEpM3IezZs0erV6/W+vXr9dFHHyk7O1vTpk3T5cuXY3E4AEAcikmA3njjDS1ZskTPP/+8vv71r2vbtm3q06ePfv/738ficACAOBT1AN24cUOVlZXKz8//30G6dVN+fr4qKiru2b+lpUWhUChiAwB0fVEP0Oeff65bt24pLS0t4vG0tDTV1dXds39xcbECgUB44x1wAPB4MP+HqGvXrlVjY2N4u3DhgvVIAIAOEPV3waWkpKh79+6qr6+PeLy+vl7BYPCe/f1+v/x+f7THAAB0clG/A0pISND48eNVWloafqy1tVWlpaXKzc2N9uEAAHEqJv8OaPXq1Vq4cKG++c1vauLEiXrzzTfV3Nys559/PhaHAwDEoZgEaN68efr3v/+tdevWqa6uTuPGjdOhQ4fueWMCAODx5XPOOesh/n+hUEiBQECNjY18EgIAxKGH/Tlu/i44AMDjiQABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDRw3oAAPDik08+8bwmPz+/Xcc6deqU5zUDBw5s17EeR9wBAQBMECAAgImoB+i1116Tz+eL2EaOHBntwwAA4lxMXgN66qmn9P777//vID14qQkAECkmZejRo4eCwWAsvjUAoIuIyWtAZ8+eVUZGhoYOHarnnntO58+fv+++LS0tCoVCERsAoOuLeoBycnK0Y8cOHTp0SFu3blVNTY0mT56spqamNvcvLi5WIBAIb5mZmdEeCQDQCfmccy6WB2hoaNCQIUP0xhtvaPHixfc839LSopaWlvDXoVBImZmZamxsVFJSUixHAxCH+HdAnV8oFFIgEHjgz/GYvzugf//+evLJJ1VdXd3m836/X36/P9ZjAAA6mZj/O6CrV6/q3LlzSk9Pj/WhAABxJOoBWrNmjcrLy/XPf/5Tf/3rXzV79mx1795dzz77bLQPBQCIY1H/K7iLFy/q2Wef1ZUrVzRw4EA9/fTTOnbsGH8vCgCIEPUA7d69O9rfsks4e/as5zX//e9/Pa+ZOHGi5zVAPDl+/LjnNXl5eTGYBI+Kz4IDAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEzE/BfS4bbS0lLPaz799FPPa/gwUsST9vxC5vZ8sO8//vEPz2sQe9wBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwASfht1BNm/e7HlNQUFBDCYBOo+rV696XlNcXOx5zYsvvuh5jSQNHDiwXevwcLgDAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBM8GGkHeTWrVvWIwCdzrJlyzrkOKNGjeqQ48Ab7oAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABN8GGk71NbWel7zr3/9KwaTAPHtP//5T4cc5zvf+U6HHAfecAcEADBBgAAAJjwH6OjRo5oxY4YyMjLk8/m0f//+iOedc1q3bp3S09PVu3dv5efn6+zZs9GaFwDQRXgOUHNzs7Kzs1VSUtLm85s2bdLmzZu1bds2HT9+XH379tW0adN0/fr1Rx4WANB1eH4TQmFhoQoLC9t8zjmnN998U6+88opmzpwpSXr77beVlpam/fv3a/78+Y82LQCgy4jqa0A1NTWqq6tTfn5++LFAIKCcnBxVVFS0uaalpUWhUChiAwB0fVENUF1dnSQpLS0t4vG0tLTwc3crLi5WIBAIb5mZmdEcCQDQSZm/C27t2rVqbGwMbxcuXLAeCQDQAaIaoGAwKEmqr6+PeLy+vj783N38fr+SkpIiNgBA1xfVAGVlZSkYDKq0tDT8WCgU0vHjx5WbmxvNQwEA4pznd8FdvXpV1dXV4a9ramp06tQpJScna/DgwVq5cqV+8Ytf6IknnlBWVpZeffVVZWRkaNasWdGcGwAQ5zwH6MSJE3rmmWfCX69evVqStHDhQu3YsUMvvfSSmpubtXTpUjU0NOjpp5/WoUOH1KtXr+hNDQCIe54DNHXqVDnn7vu8z+fTxo0btXHjxkcarDP785//7HnNtWvXYjAJ0Hk0Nzd7XvP3v/89BpPca8CAAR1yHHhj/i44AMDjiQABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACY8fxo2pDNnznTIccaNG9chxwGi4Wc/+5nnNbW1tZ7XjB071vOahIQEz2sQe9wBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAm+DDSTiwnJ8d6BHQiLS0tntdUVla261i//e1vPa/Zs2dPu47l1ebNmz2v6dWrVwwmwaPiDggAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMMGHkXZiDQ0N1iNEXW1trec1ra2tnteUl5d7XiNJNTU1ntfcuHHD85otW7Z4XnPr1i3Pa/r27et5jSQVFBR4XtOeD/y8efOm5zWjRo3yvAadE3dAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJPoy0Hfr06eN5jc/n87zm+9//vuc1I0aM8LymI1VUVHhe45zzvKZHj/Zd2v369fO8Jicnx/OaNWvWeF4zefJkz2vGjRvneY3Uvg8xzczM9LymubnZ85qBAwd6XoPOiTsgAIAJAgQAMOE5QEePHtWMGTOUkZEhn8+n/fv3Rzy/aNEi+Xy+iG369OnRmhcA0EV4DlBzc7Oys7NVUlJy332mT5+uS5cuhbddu3Y90pAAgK7H8yu1hYWFKiws/NJ9/H6/gsFgu4cCAHR9MXkNqKysTKmpqRoxYoSWL1+uK1eu3HfflpYWhUKhiA0A0PVFPUDTp0/X22+/rdLSUv3qV79SeXm5CgsL7/v77IuLixUIBMJbe97KCQCIP1H/d0Dz588P/3nMmDEaO3ashg0bprKyMuXl5d2z/9q1a7V69erw16FQiAgBwGMg5m/DHjp0qFJSUlRdXd3m836/X0lJSREbAKDri3mALl68qCtXrig9PT3WhwIAxBHPfwV39erViLuZmpoanTp1SsnJyUpOTtaGDRs0d+5cBYNBnTt3Ti+99JKGDx+uadOmRXVwAEB88xygEydO6Jlnngl/fef1m4ULF2rr1q06ffq03nrrLTU0NCgjI0MFBQX6+c9/Lr/fH72pAQBxz+fa80mPMRQKhRQIBNTY2NilXg966623PK8pKyuL/iBxaMGCBZ7XDB8+vF3HysrKate6ruaPf/yj5zXf+973PK8ZOXKk5zUff/yx5zXoWA/7c5zPggMAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAICJqP9KbrRt4cKFHbIGiIaDBw92yHF++MMfdshx0DlxBwQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmODDSAGYmTNnjvUIMMQdEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADARA/rAQB0Dc45z2s+++wzz2uGDh3qeQ06J+6AAAAmCBAAwISnABUXF2vChAlKTExUamqqZs2apaqqqoh9rl+/rqKiIg0YMED9+vXT3LlzVV9fH9WhAQDxz1OAysvLVVRUpGPHjunw4cO6efOmCgoK1NzcHN5n1apVeu+997R3716Vl5ertrZWc+bMifrgAID45ulNCIcOHYr4eseOHUpNTVVlZaWmTJmixsZG/e53v9POnTv17W9/W5K0fft2jRo1SseOHdO3vvWt6E0OAIhrj/QaUGNjoyQpOTlZklRZWambN28qPz8/vM/IkSM1ePBgVVRUtPk9WlpaFAqFIjYAQNfX7gC1trZq5cqVmjRpkkaPHi1JqqurU0JCgvr37x+xb1pamurq6tr8PsXFxQoEAuEtMzOzvSMBAOJIuwNUVFSkM2fOaPfu3Y80wNq1a9XY2BjeLly48EjfDwAQH9r1D1FXrFihgwcP6ujRoxo0aFD48WAwqBs3bqihoSHiLqi+vl7BYLDN7+X3++X3+9szBgAgjnm6A3LOacWKFdq3b5+OHDmirKysiOfHjx+vnj17qrS0NPxYVVWVzp8/r9zc3OhMDADoEjzdARUVFWnnzp06cOCAEhMTw6/rBAIB9e7dW4FAQIsXL9bq1auVnJyspKQkvfDCC8rNzeUdcACACJ4CtHXrVknS1KlTIx7fvn27Fi1aJEn69a9/rW7dumnu3LlqaWnRtGnT9Jvf/CYqwwIAug5PAXqYDxvs1auXSkpKVFJS0u6hAMQfn8/neU1ra2sMJkG84LPgAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYKJdvxEVAKLhyJEjntfk5eXFYBJY4A4IAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDBh5ECiArnnPUIiDPcAQEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJvgwUgD3mDt3ruc127Zti8Ek6Mq4AwIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATPBhpADukZeX53lNa2trDCZBV8YdEADABAECAJjwFKDi4mJNmDBBiYmJSk1N1axZs1RVVRWxz9SpU+Xz+SK2ZcuWRXVoAED88xSg8vJyFRUV6dixYzp8+LBu3rypgoICNTc3R+y3ZMkSXbp0Kbxt2rQpqkMDAOKfpzchHDp0KOLrHTt2KDU1VZWVlZoyZUr48T59+igYDEZnQgBAl/RIrwE1NjZKkpKTkyMef+edd5SSkqLRo0dr7dq1unbt2n2/R0tLi0KhUMQGAOj62v027NbWVq1cuVKTJk3S6NGjw48vWLBAQ4YMUUZGhk6fPq2XX35ZVVVVevfdd9v8PsXFxdqwYUN7xwAAxCmfc861Z+Hy5cv1pz/9SR9++KEGDRp03/2OHDmivLw8VVdXa9iwYfc839LSopaWlvDXoVBImZmZamxsVFJSUntGAwAYCoVCCgQCD/w53q47oBUrVujgwYM6evTol8ZHknJyciTpvgHy+/3y+/3tGQMAEMc8Bcg5pxdeeEH79u1TWVmZsrKyHrjm1KlTkqT09PR2DQgA6Jo8BaioqEg7d+7UgQMHlJiYqLq6OklSIBBQ7969de7cOe3cuVPf/e53NWDAAJ0+fVqrVq3SlClTNHbs2Jj8BwAA4pOn14B8Pl+bj2/fvl2LFi3ShQsX9IMf/EBnzpxRc3OzMjMzNXv2bL3yyisP/XrOw/7dIQCgc4rJa0APalVmZqbKy8u9fEsAwGOKz4IDAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJjoYT3A3ZxzkqRQKGQ8CQCgPe78/L7z8/x+Ol2AmpqaJEmZmZnGkwAAHkVTU5MCgcB9n/e5ByWqg7W2tqq2tlaJiYny+XwRz4VCIWVmZurChQtKSkoymtAe5+E2zsNtnIfbOA+3dYbz4JxTU1OTMjIy1K3b/V/p6XR3QN26ddOgQYO+dJ+kpKTH+gK7g/NwG+fhNs7DbZyH26zPw5fd+dzBmxAAACYIEADARFwFyO/3a/369fL7/dajmOI83MZ5uI3zcBvn4bZ4Og+d7k0IAIDHQ1zdAQEAug4CBAAwQYAAACYIEADARNwEqKSkRF/72tfUq1cv5eTk6G9/+5v1SB3utddek8/ni9hGjhxpPVbMHT16VDNmzFBGRoZ8Pp/2798f8bxzTuvWrVN6erp69+6t/Px8nT171mbYGHrQeVi0aNE918f06dNtho2R4uJiTZgwQYmJiUpNTdWsWbNUVVUVsc/169dVVFSkAQMGqF+/fpo7d67q6+uNJo6NhzkPU6dOved6WLZsmdHEbYuLAO3Zs0erV6/W+vXr9dFHHyk7O1vTpk3T5cuXrUfrcE899ZQuXboU3j788EPrkWKuublZ2dnZKikpafP5TZs2afPmzdq2bZuOHz+uvn37atq0abp+/XoHTxpbDzoPkjR9+vSI62PXrl0dOGHslZeXq6ioSMeOHdPhw4d18+ZNFRQUqLm5ObzPqlWr9N5772nv3r0qLy9XbW2t5syZYzh19D3MeZCkJUuWRFwPmzZtMpr4PlwcmDhxoisqKgp/fevWLZeRkeGKi4sNp+p469evd9nZ2dZjmJLk9u3bF/66tbXVBYNB9/rrr4cfa2hocH6/3+3atctgwo5x93lwzrmFCxe6mTNnmsxj5fLly06SKy8vd87d/t++Z8+ebu/eveF9PvnkEyfJVVRUWI0Zc3efB+ec+7//+z/34osv2g31EDr9HdCNGzdUWVmp/Pz88GPdunVTfn6+KioqDCezcfbsWWVkZGjo0KF67rnndP78eeuRTNXU1Kiuri7i+ggEAsrJyXksr4+ysjKlpqZqxIgRWr58ua5cuWI9Ukw1NjZKkpKTkyVJlZWVunnzZsT1MHLkSA0ePLhLXw93n4c73nnnHaWkpGj06NFau3atrl27ZjHefXW6DyO92+eff65bt24pLS0t4vG0tDR9+umnRlPZyMnJ0Y4dOzRixAhdunRJGzZs0OTJk3XmzBklJiZaj2eirq5Oktq8Pu4897iYPn265syZo6ysLJ07d04//elPVVhYqIqKCnXv3t16vKhrbW3VypUrNWnSJI0ePVrS7eshISFB/fv3j9i3K18PbZ0HSVqwYIGGDBmijIwMnT59Wi+//LKqqqr07rvvGk4bqdMHCP9TWFgY/vPYsWOVk5OjIUOG6A9/+IMWL15sOBk6g/nz54f/PGbMGI0dO1bDhg1TWVmZ8vLyDCeLjaKiIp05c+axeB30y9zvPCxdujT85zFjxig9PV15eXk6d+6chg0b1tFjtqnT/xVcSkqKunfvfs+7WOrr6xUMBo2m6hz69++vJ598UtXV1dajmLlzDXB93Gvo0KFKSUnpktfHihUrdPDgQX3wwQcRv74lGAzqxo0bamhoiNi/q14P9zsPbcnJyZGkTnU9dPoAJSQkaPz48SotLQ0/1traqtLSUuXm5hpOZu/q1as6d+6c0tPTrUcxk5WVpWAwGHF9hEIhHT9+/LG/Pi5evKgrV650qevDOacVK1Zo3759OnLkiLKysiKeHz9+vHr27BlxPVRVVen8+fNd6np40Hloy6lTpySpc10P1u+CeBi7d+92fr/f7dixw3388cdu6dKlrn///q6urs56tA714x//2JWVlbmamhr3l7/8xeXn57uUlBR3+fJl69FiqqmpyZ08edKdPHnSSXJvvPGGO3nypPvss8+cc8798pe/dP3793cHDhxwp0+fdjNnznRZWVnuiy++MJ48ur7sPDQ1Nbk1a9a4iooKV1NT495//333jW98wz3xxBPu+vXr1qNHzfLly10gEHBlZWXu0qVL4e3atWvhfZYtW+YGDx7sjhw54k6cOOFyc3Ndbm6u4dTR96DzUF1d7TZu3OhOnDjhampq3IEDB9zQoUPdlClTjCePFBcBcs65LVu2uMGDB7uEhAQ3ceJEd+zYMeuROty8efNcenq6S0hIcF/96lfdvHnzXHV1tfVYMffBBx84SfdsCxcudM7dfiv2q6++6tLS0pzf73d5eXmuqqrKdugY+LLzcO3aNVdQUOAGDhzoevbs6YYMGeKWLFnS5f5PWlv//ZLc9u3bw/t88cUX7kc/+pH7yle+4vr06eNmz57tLl26ZDd0DDzoPJw/f95NmTLFJScnO7/f74YPH+5+8pOfuMbGRtvB78KvYwAAmOj0rwEBALomAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMDE/wMXNIyPwE63lgAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "all_values= data_list[2].split(',')\n", + "image_array= np.asfarray(all_values[1:]).reshape((28,28))\n", + "plt.imshow(image_array, cmap='Greys', interpolation='None')" + ] + }, + { + "cell_type": "markdown", + "id": "5011b407", + "metadata": {}, + "source": [ + "## What the NN sees?" + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "id": "1f920b2d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[]" + ] + }, + "execution_count": 23, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAGdCAYAAAA44ojeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA3hUlEQVR4nO3de3TU9Z3/8dfMIAGFJHJLSAdGMKlovdTlEqO1UswxKtJa2T3VpRraqUnXsCtqrdJubV3b0tOes23tcdFICj2nXqp7hCpVuxRGrDZyq1SwFRNkSr6VgJUlAVoCZD6/P/hlNpPrTDIz38s8H+d8z0nme5nP5/ud+X7f87n6jDFGAAAADuK3OwEAAAA9EaAAAADHIUABAACOQ4ACAAAchwAFAAA4DgEKAABwHAIUAADgOAQoAADAcUbYnYChiMViev/99zV27Fj5fD67kwMAAJJgjNGRI0dUUlIiv3/gMhJXBijvv/++pkyZYncyAADAELS0tCgYDA64jSsDlLFjx0o6ncH8/HybUwMAAJLR3t6uKVOmxJ/jA3FlgNJVrZOfn0+AAgCAyyTTPINGsgAAwHEIUAAAgOMQoAAAAMchQAEAAI5DgAIAAByHAAUAADgOAQoAAHAcAhQAAOA4BCgAAMBxCFAAAIDjEKAAAIbEsixFIhFZlmV3UobMC3nwKgIUAEDKGhoaFAqFNG/ePIVCITU0NNidpJR5IQ9e5jPGGLsTkar29nYVFBSora2NyQJhK8uy1NTUpLKyskGnDs/2/skeu7/tktl/sG0yvT4V6TxWuo7p1v0ty1IoFFIsFou/FggEFI1GU06H2/OQic+Vl6X0/DYu1NbWZiSZtrY2u5OCHLZy5Urj9/uNJOP3+83KlSsds3+yx+5vu2T2H2ybTK9PRktLi9m4caP5wQ9+MOxj9eTk65/p/Tdu3Ggk9VoikUhO5SEdn9Fck8rzmwAFGIKWlpb4jalrCQQCpqWlxfb9kz12f9tt2bJl0P0He49Mr++ejo0bN/Z53ro/PHoufr8/6XOd6vln/+TYnQa7989VqTy/aYMCDEFTU1NC0bAkdXZ2qrm52fb9kz12f9u99tprg+4/2Htker00cPsBy7JUU1PT6xhdYrGYGhsb+1yXDCdf/2zsHwwGVV9fr0AgIOl01chjjz2WUhWH2/Mw3PdHElKJfL773e+aWbNmmTFjxpiJEyeaz3zmM+add95J2Oaqq67q9WultrY2YZs///nP5vrrrzejR482EydONF/5ylfMyZMnk04HJSiwm92/vnK9BGWw9f0V33dfnnnmmaTOdbavnxv2736cSCQypFIDt+eBEpShyVgVT1VVlVm1apXZtWuX2bFjh7n++uvN1KlTzdGjR+PbXHXVVeb22283+/fvjy/dE3Lq1Clz4YUXmsrKSvPmm2+aF1980UyYMMEsW7YsIxkEMmXlypUmEAjEb0xDqT/P1P7JHru/7ZLZf7BtMrl+sPYDfT08ei5dwVx/VUSDcfL1z8b+6eD2PNj9/m6UtTYoBw8eNJLMpk2b4q9dddVV5s477+x3nxdffNH4/X7T2toaf23FihUmPz/fdHR0JPW+BChwiuH8gsz0/skeu7/tktl/sG0ytT6ZX6/dHx4+ny9hW5/PZ6qrq9PSCNep1z8b+6eD2/Ng9/u7TSrP72F1M25ublZZWZl27typCy+8UJI0d+5cvf322zLGqLi4WAsWLNA3vvENnXnmmZKkBx54QM8//7x27NgRP87evXs1ffp0/f73v9ell17a6306OjrU0dER/7+9vV1TpkyhmzGQwxoaGlRbW6vOzs54+4FwOJywjWVZam5u1llnnaXLLrus3zYp0tC7yQJIXirdjEcM9U1isZiWLl2qK664Ih6cSNI///M/KxQKqaSkRG+99Zbuu+8+7d69W88995wkqbW1VUVFRQnH6vq/tbW1z/davny5HnzwwaEmFYBDDWcMiXA4rKqqKjU3N6u0tLTP/YPBoILBoCKRyIDBifR/DRwJUABnGHKAUldXp127dum1115LeL2mpib+90UXXaTJkyfr6quv1p49e3TuuecO6b2WLVumu+++O/5/VwkKAPdqaGiI97Tx+/2qr6/vVQIymK4AZDBlZWXy+/2DlqCUlpam9P4AMmdI3YyXLFmidevWKRKJDHpzKC8vl6R416vi4mIdOHAgYZuu/4uLi/s8Rl5envLz8xMWAO7VsxtwLBZTbW1txuZD6atLaXV19bC6yQLIrJQCFGOMlixZojVr1mjjxo2aNm3aoPt0tTWZPHmyJKmiokI7d+7UwYMH49usX79e+fn5uuCCC1JJDgCXsmMMiXA4rGg0qkgkomg0qtWrVyf8n2rpDYDMSqmR7B133KEnn3xSv/zlL3XeeefFXy8oKNDo0aO1Z88ePfnkk7r++us1fvx4vfXWW7rrrrsUDAa1adMmSadvQh//+MdVUlKi73//+2ptbdWtt96qL33pS/rud7+bVDqYiwe5wMtzfKRzLhcA7pGxuXjUz3gCq1atMsYYs2/fPvPJT37SjBs3zuTl5ZnS0lJz77339upOFI1GzXXXXWdGjx5tJkyYYO655x4GagO6yYU5PhhDAsg9WetmbBdKUOBluVS60NUNuL9eOAPt59XSJcDLUnl+MxcP4DC5NMdHMBjU3LlzUwoyBpqDB4B3EKAADtPVJbY7usCelu3ePwDsQ4ACOEw6Zor1qv5Kl4YzMzEAZxryQG0AMieZUVJz0bZt2/p8/eabb1Z7e/ugXYVpuwK4B41kAbhCX42HuxusIXE6Rq4FMDw0kgXgOX1V73Q3UENi2q4A7kOAAsAV+mo83N1ADYlzqWcU4BUEKABcoWfjYZ/PFw9Y+mtIbFmWIpGIxowZQ88owGVoJAvANXo2HpbUb0Pinm1Obr31Vv385z9XZ2cnPaMAF6CRLADP6W803sbGRh07doyeUYBNUnl+U4ICwHP6a3Ny7NgxzZ07155EAUgJbVAAuFJX+5K+euIwGi/gfgQoAFxnsPl4kh2Nd6AgB4C9aIMCwFVSme15oNmSGbgNyD4GaoNr8AsWqUplTJP+Zktm4DbA+QhQYJvBiumReW4MENPRvoSB2wDnI0CBLfgFaz+3BojpmO2ZRrSA8xGgwBb8grWX2wPEcDisaDSqSCSiaDSactuRdAQ5ADKLcVBgi65fsD0bOvILNjsGChDd8pAOBoPDSmvPUWndkm8gV1CCAlvwC9ZeVHGc1l8jWgD2I0CBbYZbTI+hI0AE4HSMgwLksIHGCQGAdGMuHgBJGW47DgDIFKp4AACA4xCgAAAAxyFAAQAAjkOAAgAAHIcABcgQN85zAwBOQYACZIBb57kBAKdgHBQgzSzLUigU6jWMfzQapUsvgJyWyvObEhSgH0OtomEixNxCVR6QGQQoQB+GU0XDPDe5g6o8IHOo4gF6SEcVTUNDg2pra9XZ2Rmf54a5hryFqjwgdQx1DwzDQFU0yT54wuGwqqqqmOfGw9LxOQHQPwIUoIeuKpqev4xTraJhnhtvS9fnBEDfaIMC9BAMBlVfX69AICBJ8Soago2+5WojUT4nQGbRBgXoh2VZVNEMoqGhQTU1NYrFYvL7/aqvr8+5tjZ8ToDkpfL8JkABMCQ0EgWQKsZBAZBxjPcCIJMIUAAMCeO9AMgkAhQAQ0IjUQCZRBsUAMNCI1EAyWKgNgBZw3gvADKBKh7kvFwdxwMAnIwABTmNyd4AwJlog4KcxTgeAJBdjIMCJIFxPADAuWgki5zFZG+5wbIsvfDCC9q/f78WLFig2bNn250kAEmgBAU5i3E8vK+hoUFTpkzRHXfcoYceekhz5szR4sWL7U4WgCTQBgU5j3E8vMmyLE2ZMqXPdVu2bKEkBbAB46AAKWAcD29qamrqd93rr79OgAI4HFU8ADyprKys33VXXHFFFlMCYCgIUAB4UjAY1MqVK3u9Xl1dTekJ4AK0QQHgaZZlad26dWptbdX8+fMJTgAbpfL8JkABAABZkbGB2pYvX67Zs2dr7NixmjRpkm688Ubt3r07YZvjx4+rrq5O48eP15gxY7Rw4UIdOHAgYZt9+/Zp/vz5OvPMMzVp0iTde++9OnXqVCpJAQAAHpZSgLJp0ybV1dXpjTfe0Pr163Xy5Eldc801OnbsWHybu+66Sy+88IKeffZZbdq0Se+//75uuumm+PrOzk7Nnz9fJ06c0O9+9zv97Gc/0+rVq/XAAw+kL1cAAMDVhlXF88EHH2jSpEnatGmTPvnJT6qtrU0TJ07Uk08+qX/8x3+UJL3zzjs6//zz1djYqMsuu0wvvfSSbrjhBr3//vsqKiqSJD366KO677779MEHH2jkyJGDvi9VPAAAuE/W5uJpa2uTJI0bN06StH37dp08eVKVlZXxbWbMmKGpU6eqsbFRktTY2KiLLrooHpxIUlVVldrb2/X222/3+T4dHR1qb29PWAAAgHcNOUCJxWJaunSprrjiCl144YWSpNbWVo0cOVKFhYUJ2xYVFam1tTW+TffgpGt917q+LF++XAUFBfGlv9EhgVxjWZYikYgsy7I7Ka7BOQPcYcgBSl1dnXbt2qWnn346nenp07Jly9TW1hZfWlpaMv6egNM1NDQoFApp3rx5CoVCamhosDtJjsc5A9xjSAHKkiVLtG7dOkUikYQhwouLi3XixAkdPnw4YfsDBw6ouLg4vk3PXj1d/3dt01NeXp7y8/MTFiCXWZalmpqa+EzMsVhMtbW1lAoMgHMGuEtKAYoxRkuWLNGaNWu0ceNGTZs2LWH9zJkzdcYZZ2jDhg3x13bv3q19+/apoqJCklRRUaGdO3fq4MGD8W3Wr1+v/Px8XXDBBcPJC5Azmpqa4g/aLp2dnWpubrYpRc7HOQPcJaXJAuvq6vTkk0/ql7/8pcaOHRtvM1JQUKDRo0eroKBA4XBYd999t8aNG6f8/Hz967/+qyoqKnTZZZdJkq655hpdcMEFuvXWW/X9739fra2t+vd//3fV1dUpLy8v/TkEPKisrEx+vz/hgRsIBFRaWmpjqpyNcwa4S0olKCtWrFBbW5vmzp2ryZMnx5df/OIX8W1++MMf6oYbbtDChQv1yU9+UsXFxXruuefi6wOBgNatW6dAIKCKigp9/vOf12233ab/+I//SF+uAI8LBoOqr69XIBCQdPp79dhjjzEr8wA4Z4C7MNQ94GKWZam5uVmlpaU8aJPEOQPsk8rzO6UqHgDOEgwGecimiHMGuMOwBmoDAADIBAIUAADgOAQoAADAcQhQAACA4xCgAAAAxyFAAQAAjkOAAgAAHIcABQAAOA4BCgAAcBwCFAAA4DgEKAAAwHEIUAAbWZalSCQiy7LsTgoAOAoBCmCThoYGhUIhzZs3T6FQSA0NDXYnCQAcw2eMMXYnIlWpTNcMOJFlWQqFQorFYvHXAoGAotEoM+0C8KxUnt+UoAA2aGpqSghOJKmzs1PNzc02pQgAnIUABa7m1jYcZWVl8vsTv36BQEClpaU2pQgAnIUABa7l5jYcwWBQ9fX1CgQCkk4HJ4899hjVOwDw/9EGBa7klTYclmWpublZpaWlrkq3E1iWpaamJpWVlXHuAJegDQo8zyttOILBoObOnZuTD9jhVM+5ufQMQHIIUOBKtOFwt+EEGJZlqaamJh6gxmIx1dbWZqwd0tatW7V06VKFw2GtWLHCde2dALciQIEr0YbDvYYbYGSz9Gzx4sWaM2eOfvzjH+unP/2p7rjjDk2dOpUSGyALCFDgWuFwWNFoVJFIRNFoVOFw2O4kIQnDDTCyVXq2detW/exnP+v1ujFGNTU1lKQAGUaAAlfL5TYcbjXcACNbpWe//e1v+10Xi8Vc194JcBsCFABZlY4AIxulZ1deeWW/6/x+P+2dgAyjmzEAW7ihi/XixYt7VfP4fD49/vjjVCkCQ5DK85sABQAGsHXrVj311FM6cuSIZs6cqRtuuMGxARXgdKk8v0dkKU0A4EqzZ8/W7Nmz7U4GkHNogwIAAByHAAUAADgOAQoAAHAcAhQAsMlw5iNif/v3R4YZF2prazOSTFtbm91JAYAhWblypfH7/UaS8fv9ZuXKlezvov0xNKk8v+lmDABZZlmWQqFQwpD/gUBA0Wg0qS7M7G/v/hi6VJ7fVPEAQJYNdz4i9rd3f2QHAQoAZNlw5yNif3v3R3YQoABAlg13PiL2t3d/ZAdtUADAJsOdj4j97d0fqWMuHgAA4Dg0kgWUnjEO7B5nwe797T4+gNxFgAJPamhoUCgU0rx58xQKhdTQ0JCxY/T3kB5uGuze3+7jA8htVPHAc9IxxkGyx2hoaFBNTY1isZj8fr/q6+sVDodtH6ch0+M8MI4EgKGgigc5LR1jHCRzDMuy4sGJJMViMdXW1sqyLNvHacj0OA+MIwEg0whQ4DnpGOMgmWMM9JC2e5yGTI/zwDgSADKNAAWek44xDpI5xkAPabvHaUjHORioAWwyx3d6A18ADpfBOYEyhskCkYyWlhYTiURMS0tLxo6xcuVKEwgEjCQTCAR6TTg23DTYtX+yE6n1d3wmcgPQFyYLBLLIa4M92d1Alwa4gHfRSBbIomAwqLlz53rm4Wl3A10a4AKQCFAA9GB3A10a4AKQCFAA9GB3A10mcgMgMVAbgH7YPRGb19r2AGCyQAAA4EA0kgWSxFgbgLs5Ybwd7iOZQYCCnMVkd4C7OWFCTe4jmUMVD3ISY20A7uaE8Xa4j6Quo1U8r776qhYsWKCSkhL5fD6tXbs2Yf3ixYvl8/kSlmuvvTZhm0OHDmnRokXKz89XYWGhwuGwjh49mmpSgCFjrA3A3Zww3g73kcxKOUA5duyYLrnkEj3yyCP9bnPttddq//798eWpp55KWL9o0SK9/fbbWr9+vdatW6dXX31VNTU1qaceGCLG2nCvZOv7aRfgbU4Yb4f7SIYNZ0x9SWbNmjUJr1VXV5vPfOYz/e7zxz/+0UgyW7dujb/20ksvGZ/PZ/7yl78k9b7MxYN0GGweHThPsnP0MJdPbhjudzgd9wDuI6nJ2lw8Pp9Pa9as0Y033hh/bfHixVq7dq1Gjhyps88+W/PmzdO3v/1tjR8/XpL005/+VPfcc4/+93//N77PqVOnNGrUKD377LP67Gc/2+t9Ojo61NHREf+/vb1dU6ZMoQ0Kho2xNtwj2fp+2gXkFieMt8N9JHmptEEZke43v/baa3XTTTdp2rRp2rNnj772ta/puuuuU2NjowKBgFpbWzVp0qTERIwYoXHjxqm1tbXPYy5fvlwPPvhgupMKKBgMckNxiYHq+7tfw2S3gzcM9zucjnsA95HMSHuAcvPNN8f/vuiii3TxxRfr3HPP1SuvvKKrr756SMdctmyZ7r777vj/XSUoAHJHV31/z5KRnvX9yW4HwNkyPg7K9OnTNWHChHir5uLiYh08eDBhm1OnTunQoUMqLi7u8xh5eXnKz89PWADklmTn6GEuH8Ab0l6C0pNlWfrwww81efJkSVJFRYUOHz6s7du3a+bMmZKkjRs3KhaLqby8PNPJAeBi4XBYVVVVg9b3J7sdAOdKOUA5evRoQh/vvXv3aseOHRo3bpzGjRunBx98UAsXLlRxcbH27Nmjr371qyotLVVVVZUk6fzzz9e1116r22+/XY8++qhOnjypJUuW6Oabb1ZJSUn6cgbAk5Kt76ddAOBuKffieeWVV/SpT32q1+vV1dVasWKFbrzxRr355ps6fPiwSkpKdM011+ihhx5SUVFRfNtDhw5pyZIleuGFF+T3+7Vw4UI9/PDDGjNmTFJpYCRZIDdZlqWmpiaVlZURfAAuxGzGADynoaFBNTU1isVi8vv9qq+vVzgcHtKxCHQAezCbMQBPsSwrHpxIUiwWU21t7ZBGiWVyN8AdCFAAOF665jxJZ6ADILMIUAA4XrrmPGFyN8A9CFAAOF66xjZhcjfAPQhQALhCOBxWNBpVJBJRNBodUgNZBnED3INePAByDpO7AfawdbJAAHA6BnEDnI8qHgCwkWVZikQi9CQCeiBAAQCbMCYL0D/aoACADSzLUigUSuj2HAgEFI1GqX6CZzGSLAA4HGOyAAMjQAEAGzAmCzAwAhQAsAFjsgADow0KANiIMVmQSxgHBQBcgjFZgL5RxQMAAByHAAUAADgOAQoAAHAcAhQAAOA4BCgAAMBxCFAAAIDjEKDAs5glFgDciwAFnsQssQDgbowkC89x2yyxlmWpqalJZWVljkwfAKQLsxkjp7lpllhKegCgbwQo8By3zBJrWZZqamriwVQsFlNtbS1tZgBABCjwILfMEuumkh4AyDYmC4QnhcNhVVVVOXqW2K6Snp5tZZxW0gMAdqAEBZ4VDAY1d+5cRwYnkntKegDADvTigat5oQeMZVkZKelJx7nxwvkF4Bz04kFO8EoPmEyU9KTj3Hjl/AJwJ0pQ4EpuG+skm9Jxbji/p1GCBKQXJSjwPHrA9C8d54bzSwkSYDcCFLiSW8Y6sUM6zk2un1/GqAHsR4ACV6IHTP/ScW5y/fxSggTYjzYocLVM9YDxgnScG6ed32y1CaENDpAZqTy/CVAAuEJDQ0O82sXv96u+vl7hcDij71dbW6vOzs54CVIm3w/IBQQoADzFrhINp5UgAW6XyvOboe4BON5AbUIyGTgEg0ECE8AmNJIF4Hi53qsIyEUEKAAcL9d7FQG5iDYoAFyDNiGAu9EGBYAn0SYEyB1U8QAAAMchQAEAAI5DgAIAAByHAAUAADgOAQoAAHAcAhQAAOA4BCgAAMBxCFAAAIDjEKAAAADHIUABAACOQ4ACAENgWZYikYgsy7I7KYAnEaAAQIoaGhoUCoU0b948hUIhNTQ02J0kwHNSDlBeffVVLViwQCUlJfL5fFq7dm3CemOMHnjgAU2ePFmjR49WZWWlmpqaErY5dOiQFi1apPz8fBUWFiocDuvo0aPDyggAZINlWaqpqVEsFpMkxWIx1dbWUpICpFnKAcqxY8d0ySWX6JFHHulz/fe//309/PDDevTRR7V582adddZZqqqq0vHjx+PbLFq0SG+//bbWr1+vdevW6dVXX1VNTc3QcwEAWdLU1BQPTrp0dnaqubnZphQB3uQzxpgh7+zzac2aNbrxxhslnS49KSkp0T333KOvfOUrkqS2tjYVFRVp9erVuvnmm/WnP/1JF1xwgbZu3apZs2ZJkl5++WVdf/31sixLJSUlg75ve3u7CgoK1NbWpvz8/KEmHwBSZlmWQqFQQpASCAQUjUYVDAZtTBngfKk8v9PaBmXv3r1qbW1VZWVl/LWCggKVl5ersbFRktTY2KjCwsJ4cCJJlZWV8vv92rx5c5/H7ejoUHt7e8ICANnU1ShWkurr6xUIBCSdDk4ee+wxghMgzdIaoLS2tkqSioqKEl4vKiqKr2ttbdWkSZMS1o8YMULjxo2Lb9PT8uXLVVBQEF+mTJmSzmQDwIB6NoqVpGg0qkgkomg0qnA4bHMKAe9xRS+eZcuWqa2tLb60tLTYnSQAOaK/RrGSNHfuXEpOgAxJa4BSXFwsSTpw4EDC6wcOHIivKy4u1sGDBxPWnzp1SocOHYpv01NeXp7y8/MTFgDIBhrFZpabx5Nxc9rdIK0ByrRp01RcXKwNGzbEX2tvb9fmzZtVUVEhSaqoqNDhw4e1ffv2+DYbN25ULBZTeXl5OpMDAMNWVlYmvz/xVhkIBFRaWmpTirzDzePJuDntbpFygHL06FHt2LFDO3bskHS6YeyOHTu0b98++Xw+LV26VN/+9rf1/PPPa+fOnbrttttUUlIS7+lz/vnn69prr9Xtt9+uLVu26PXXX9eSJUt08803J9WDBwCyKRgM0ig2A9w8noyb0+4mI1LdYdu2bfrUpz4V///uu++WJFVXV2v16tX66le/qmPHjqmmpkaHDx/WJz7xCb388ssaNWpUfJ8nnnhCS5Ys0dVXXy2/36+FCxfq4YcfTkN2ACD9wuGwqqqq1NzcrNLSUoKTbizLUlNTk8rKylI6LwNVnTn9/Lo57W4yrHFQ7MI4KABgv4aGhnhJgt/vV319fdI9mtw8noyb024328ZBAQDkhuFWc7i56szNaXcTSlAAACmLRCKaN29en6/PnTs36eNYluXaqjM3p90uqTy/U26DAgDpNNQ2DG57T6/p6t3Us5oj1d5NwWDQtdfAzWl3A6p4ANjGjq6adA9ND6o5kGlU8QCwhR0NDWncmH5UcyAVVPEAcDw7umrSPTT9qOZAplDFA8AWdozQyqiwgHsQoACwhR1tGGg3AbgHbVAAZF33XjSSst6GgXYTgD1ogwLAsVIZfTRT3YFpNwE4H1U8ALImldFH6Q4M5DYCFABZM1Avmu6YLRYAAQqArEm2F02ygQwA7yJAAZA1yfaioTswAAIUAFkVDocVjUYViUQUjUb7bCBLd2AAdDOGYzGhG+gODHhLKs9vSlDgSPTggHS6JGXu3LkEJ0AOogQFjsOEbgDgTZSgwNXowQEAIECB49CDAwBAgALHoQcHAIA2KHAsenAAgLcwWSA8gQndACB3UcUDAAAchwAFAAA4DgEKAABwHAIUAADgOAQoAADAcQhQAACA4xCgAAAAxyFAAQAAjkOAAgAAHIcABQAAOA4BCmxlWZYikYgsy7I7KcgAri+AoSJAgW0aGhoUCoU0b948hUIhNTQ02J0kpBHXF8BwMJsxbGFZlkKhkGKxWPy1QCCgaDTKBIEewPUF0JdUnt+UoMAWTU1NCQ8vSers7FRzc7NNKUI6cX0BDBcBCmxRVlYmvz/x4xcIBFRaWmpTipBOXF8Aw0WAAlsEg0HV19crEAhIOv3weuyxxyj+9wiuL4Dhog0KbGVZlpqbm1VaWsrDy4O4vgC6S+X5PSJLaQL6FAwG+31wWZalpqYmlZWV8XBzqYGuLwAMhCoeOBJdVAEgt1HFA8ehiyoAeBPdjOFqdFEFABCgwHHoogoAIECB49BFFQBAGxQ4Fl1UgcENt7cbveWQTbRBgScEg0HNnTuXmybQj+H2dqO3HJyMEhTA5fgFnD1OOtfD7e1GbznYgRIUIEfwCzh7nHauh9vbjd5ycDpKUACX4hdw9jjxXFOCAjeiBAXIAfwCzh4nnuvh9najtxycjhIUwKX4BZw9Tj7Xw+3tRm85ZBMlKEAO4Bdw9jj5XA+3txu95eBUaQ9QvvWtb8nn8yUsM2bMiK8/fvy46urqNH78eI0ZM0YLFy7UgQMH0p0MICeEw2FFo1FFIhFFo1GFw2G7k+RZnGsgu0Zk4qAf+9jH9Jvf/Ob/3mTE/73NXXfdpV/96ld69tlnVVBQoCVLluimm27S66+/nomkAJ4XDAb59ZslnGsgezISoIwYMULFxcW9Xm9ra1NDQ4OefPJJzZs3T5K0atUqnX/++XrjjTd02WWXZSI5AADAZTLSBqWpqUklJSWaPn26Fi1apH379kmStm/frpMnT6qysjK+7YwZMzR16lQ1NjZmIikAPMKyLEUiEVmWZXdSAGRB2gOU8vJyrV69Wi+//LJWrFihvXv36sorr9SRI0fU2tqqkSNHqrCwMGGfoqIitba29nvMjo4Otbe3JywAcofTBkkDkHlpr+K57rrr4n9ffPHFKi8vVygU0jPPPKPRo0cP6ZjLly/Xgw8+mK4kAnARy7JUU1MT7+Ibi8VUW1urqqoq2oMAHpbxbsaFhYX66Ec/qubmZhUXF+vEiRM6fPhwwjYHDhzos81Kl2XLlqmtrS2+tLS0ZDjVAJzCiYOkAci8jAcoR48e1Z49ezR58mTNnDlTZ5xxhjZs2BBfv3v3bu3bt08VFRX9HiMvL0/5+fkJC+B1tLk4raysTD6fL+E1v9+v0tJSm1IEIBvSHqB85Stf0aZNmxSNRvW73/1On/3sZxUIBHTLLbeooKBA4XBYd999tyKRiLZv364vfOELqqiooAcP0A1tLgbmwgGwAaQo7W1QLMvSLbfcog8//FATJ07UJz7xCb3xxhuaOHGiJOmHP/yh/H6/Fi5cqI6ODlVVVem//uu/0p0MwLVoc5GoqampV0BijFFzc3NOng8gV6Q9QHn66acHXD9q1Cg98sgjeuSRR9L91oAnDNTmIhcfyGVlZfL7/b3mwaGKB/A25uIBHKbrgdxdLj+QnTwPDoDMIUABHIYHcm/MgwPkHp9xYWuzVKZrBtzKsiw1NzertLQ0p4MTAN6RyvM7I3PxABg+JqYDkMuo4kHOY7wRd+P6Ad5EgALbDfcBM5z9GW/E3bh+gHfRBgW2amhoiI/54ff7VV9fn1IDyOHsb1mWQqFQr+6r0WiUqhUX4PoB7pPK85sSFNimvwHJki0JGe7+zPHiblw/wNsIUGCb4T5ghrs/4424G9cP8DYCFNhmuA+Y4e7PeCPuxvUDvI0ABbYZ7gMmHQ8oBgBzN64f4F00koXthjsgGQOaAYA7MFAbXGW4A5IxoBkAeA9VPAAAwHEIUAAAgOMQoAAAAMchQAEAAI5DgAIAAByHAAXIccwGDMCJCFCAHMZswACcioHagByV67MBW5alpqYmlZWV5UR+ASdgNmMgS9xcPZLLswFTcgQ4HwEKMERuf8jl6mzAlmWppqYmHpzFYjHV1ta6MsgEvIwABRgCLzzkcnU24FwuOQLchLl4gCEY6CHnpgd8OBxWVVVVTk222FVy1LPtjddLjgC3oQQFGAIvVY8Eg0HNnTs3J4ITKXdLjgC3IUABhsALDzk3N/AdrnA4rGg0qkgkomg0qnA4bHeSAPRAN2NgGCzLcmX1SENDQ7wNjd/vV319PQ9pABmXyvObAAXIMbk+/gkA+zAOCoB+0YsFgBsQoAA5xksNfAF4FwEKkGO80MAXgPfRBgXIUW5t4AvAvVJ5fjNQG5CjgsEggQkAx6KKBwAAOA4BCoBhyeUB3wBkDgEKgCFz+4zOAJyLRrIAhoQB3wCkioHaAGTccAZ8o1oIwGAIUAAMyVAHfKNaCEAyCFAADMlQBnyzLCs+SaEkxWIx1dbWUpICoBfGQQEwZOFwWFVVVUkP+DZQtRDtVgB0R4ACYFhSGfCtq1qoZ8Na5gEC0BNVPACyhnmAACSLbsYAso55gIDcxFw8AByNeYAADIYqHgAA4DgEKAAAwHEIUAAAgOMQoAAAAMchQAGQk5gPCHA2AhQAOYf5gADnYxwUADnFsiyFQqFeo9lGo1G6PgMZlsrzmxIUADlloPmAADgHAQpyFm0QclPXfEDdMR8Q4Dy2BiiPPPKIzjnnHI0aNUrl5eXasmWLnclBDqENQu5iPiDAHWxrg/KLX/xCt912mx599FGVl5frRz/6kZ599lnt3r1bkyZNGnDfTLZB2bp1q5544gkdOXJEEyZM0F//+lcdO3ZMZ5111pD/l+SIYzklHd3/z8vL04IFCyRJTzzxhFpbWxPWTZ8+Xe+99546Ojr6PNbYsWO1aNGi+P7dr9vYsWNVWVmplpYWbdu2Lb5vXl6eHn30UXX/6Pt8Pn3hC19IyFfP/fvK86xZszRr1ixt27ZNu3fvVl5eXnx9cXFxfP+e67rncc6cOfrb3/6m5uZm7dmzZ9Bj90xj9/z3PH/9HXvChAnq6OjQeeedpwULFmj//v19nr/+jt2VjlmzZsX3f+GFFxKuU3Fx8YDXZtGiRZo9e7Ysy9Lvfve7Xmk899xzVVpaqjPPPFNbtmzR5MmTtWDBgl6BRM/9u85P9/z1FXx0zQd01llnadu2bQmfkb7S2dc9ovs16u8zMtB3ZqDPTzLfv8Gu8XC+f8mkY6DvXzLp6P4Z6fn5SSYdPT8jXfsne3/q+ox0v4ap3Ou6vgPD+QzYfQ/u71hd5/byyy9Pa/Ce0vPb2GTOnDmmrq4u/n9nZ6cpKSkxy5cvH3TftrY2I8m0tbWlNU3V1dVGEgsLS5aWyy+/3Ph8vqS39/l8ZuXKlfHv7MqVKwfdv+c+3a1cuTKp962uruYewZKTy0Dfn6FI5fltS4DS0dFhAoGAWbNmTcLrt912m/n0pz/da/vjx4+btra2+NLS0pJ0BpO1ZcsW2z8ILCwsgy9+v9+0tLSYlpaWpIObrn2667qPJLusWrXK9ryzsNixBAKBXt+foUolQLGlDcpf//pXdXZ2qqioKOH1oqIitba29tp++fLlKigoiC9TpkxJe5p++9vfpv2YANIvFoupublZTU1NCdV0yezTXVNTU0rvu3bt2pS2B7zCrl5urujFs2zZMrW1tcWXlpaWtL/HlVdemfZjAkg/v9+v0tJSlZWVyefzpbRPd2VlZSm974033pjS9oBX2NXLzZYAZcKECQoEAjpw4EDC6wcOHFBxcXGv7fPy8pSfn5+wpNvs2bNVXV2d9uMC6N/ll1+edJAhnW7MXF9fr2AwqGAwqMcff3zQ/bvv010wGNTKlSuTet/q6motXryYewRyjt/vt62Xm229eMrLyzVnzhz95Cc/kXS6CHbq1KlasmSJ7r///gH3zXQvnqeeekpHjhzR+PHj9eGHH8ZbNQ/1f0mOOJZT0tH9/1GjRmn+/PmSpKeeeirei6Br3TnnnKNoNKrjx4/3eayxY8fqlltuie/f/bqNHTtW8+bNk2VZ2r59e595mDlzpm644Qbt379/0P37yvPMmTM1c+ZMbd++Xe+++65GjhwZX19cXBzfv+e67nmcNWuW/v73v6u5uVnvvffeoMfuno6e+e95/vo79vjx43XixAl99KMfHTD//R27r/P3q1/9KuE6FRcXD3htbrnllngvnsbGxl5pnD59ukpLSzV69Ght27ZNxcXFuuGGG/rsxdN9/67z0z1/A91cLcvSunXr+vyMdE9nX/eI7teov8/IQN+ZgT4/yXz/BrvGw/n+JZOOgb5/yaSj+2ek5+cnmXT0/Ix07Z/s/anrM9L9GqZyr+v6DgznM2D3Pbi/Y3Wd24qKCtt68djazbi6ulqPPfaY5syZox/96Ed65pln9M477/Rqm9ITQ90DAOA+qTy/R2QpTb187nOf0wcffKAHHnhAra2t+vjHP66XX3550OAEAAB4H5MFAgCArGCyQAAA4GoEKAAAwHEIUAAAgOMQoAAAAMchQAEAAI5DgAIAAByHAAUAADgOAQoAAHAcAhQAAOA4tg11Pxxdg9+2t7fbnBIAAJCsrud2MoPYuzJAOXLkiCRpypQpNqcEAACk6siRIyooKBhwG1fOxROLxfT+++9r7Nix8vl8aT12e3u7pkyZopaWFs/O85MLeZRyI5+5kEeJfHpJLuRRyo18DiWPxhgdOXJEJSUl8vsHbmXiyhIUv9+vYDCY0ffIz8/37IeqSy7kUcqNfOZCHiXy6SW5kEcpN/KZah4HKznpQiNZAADgOAQoAADAcQhQesjLy9M3v/lN5eXl2Z2UjMmFPEq5kc9cyKNEPr0kF/Io5UY+M51HVzaSBQAA3kYJCgAAcBwCFAAA4DgEKAAAwHEIUAAAgOMQoHTzyCOP6JxzztGoUaNUXl6uLVu22J2klLz66qtasGCBSkpK5PP5tHbt2oT1xhg98MADmjx5skaPHq3Kyko1NTUlbHPo0CEtWrRI+fn5KiwsVDgc1tGjR7OYi4EtX75cs2fP1tixYzVp0iTdeOON2r17d8I2x48fV11dncaPH68xY8Zo4cKFOnDgQMI2+/bt0/z583XmmWdq0qRJuvfee3Xq1KlsZqVfK1as0MUXXxwf/KiiokIvvfRSfL3b89ef733ve/L5fFq6dGn8NS/k9Vvf+pZ8Pl/CMmPGjPh6L+RRkv7yl7/o85//vMaPH6/Ro0froosu0rZt2+LrvXD/Oeecc3pdS5/Pp7q6OkneuJadnZ36xje+oWnTpmn06NE699xz9dBDDyXMnZO1a2lgjDHm6aefNiNHjjQ//elPzdtvv21uv/12U1hYaA4cOGB30pL24osvmq9//evmueeeM5LMmjVrEtZ/73vfMwUFBWbt2rXmD3/4g/n0pz9tpk2bZv7+97/Ht7n22mvNJZdcYt544w3z29/+1pSWlppbbrklyznpX1VVlVm1apXZtWuX2bFjh7n++uvN1KlTzdGjR+PbfPnLXzZTpkwxGzZsMNu2bTOXXXaZufzyy+PrT506ZS688EJTWVlp3nzzTfPiiy+aCRMmmGXLltmRpV6ef/5586tf/cq8++67Zvfu3eZrX/uaOeOMM8yuXbuMMe7PX1+2bNlizjnnHHPxxRebO++8M/66F/L6zW9+03zsYx8z+/fvjy8ffPBBfL0X8njo0CETCoXM4sWLzebNm817771nfv3rX5vm5ub4Nl64/xw8eDDhOq5fv95IMpFIxBjjjWv5ne98x4wfP96sW7fO7N271zz77LNmzJgx5sc//nF8m2xdSwKU/2/OnDmmrq4u/n9nZ6cpKSkxy5cvtzFVQ9czQInFYqa4uNj84Ac/iL92+PBhk5eXZ5566iljjDF//OMfjSSzdevW+DYvvfSS8fl85i9/+UvW0p6KgwcPGklm06ZNxpjTeTrjjDPMs88+G9/mT3/6k5FkGhsbjTGnAzm/329aW1vj26xYscLk5+ebjo6O7GYgSWeffbZZuXKlJ/N35MgRU1ZWZtavX2+uuuqqeIDilbx+85vfNJdcckmf67ySx/vuu8984hOf6He9V+8/d955pzn33HNNLBbzzLWcP3+++eIXv5jw2k033WQWLVpkjMnutaSKR9KJEye0fft2VVZWxl/z+/2qrKxUY2OjjSlLn71796q1tTUhjwUFBSovL4/nsbGxUYWFhZo1a1Z8m8rKSvn9fm3evDnraU5GW1ubJGncuHGSpO3bt+vkyZMJ+ZwxY4amTp2akM+LLrpIRUVF8W2qqqrU3t6ut99+O4upH1xnZ6eefvppHTt2TBUVFZ7LnyTV1dVp/vz5CXmSvHUtm5qaVFJSounTp2vRokXat2+fJO/k8fnnn9esWbP0T//0T5o0aZIuvfRSPf744/H1Xrz/nDhxQj//+c/1xS9+UT6fzzPX8vLLL9eGDRv07rvvSpL+8Ic/6LXXXtN1110nKbvX0pWTBabbX//6V3V2diZ8aCSpqKhI77zzjk2pSq/W1lZJ6jOPXetaW1s1adKkhPUjRozQuHHj4ts4SSwW09KlS3XFFVfowgsvlHQ6DyNHjlRhYWHCtj3z2dd56FrnBDt37lRFRYWOHz+uMWPGaM2aNbrgggu0Y8cOT+Svy9NPP63f//732rp1a691XrmW5eXlWr16tc477zzt379fDz74oK688krt2rXLM3l87733tGLFCt1999362te+pq1bt+rf/u3fNHLkSFVXV3vy/rN27VodPnxYixcvluSdz+v999+v9vZ2zZgxQ4FAQJ2dnfrOd76jRYsWScrus4QABa5VV1enXbt26bXXXrM7KWl33nnnaceOHWpra9N///d/q7q6Wps2bbI7WWnV0tKiO++8U+vXr9eoUaPsTk7GdP3ylKSLL75Y5eXlCoVCeuaZZzR69GgbU5Y+sVhMs2bN0ne/+11J0qWXXqpdu3bp0UcfVXV1tc2py4yGhgZdd911KikpsTspafXMM8/oiSee0JNPPqmPfexj2rFjh5YuXaqSkpKsX0uqeCRNmDBBgUCgV2vrAwcOqLi42KZUpVdXPgbKY3FxsQ4ePJiw/tSpUzp06JDjzsOSJUu0bt06RSIRBYPB+OvFxcU6ceKEDh8+nLB9z3z2dR661jnByJEjVVpaqpkzZ2r58uW65JJL9OMf/9gz+ZNOV28cPHhQ//AP/6ARI0ZoxIgR2rRpkx5++GGNGDFCRUVFnslrd4WFhfroRz+q5uZmz1zPyZMn64ILLkh47fzzz49XZXnt/vPnP/9Zv/nNb/SlL30p/ppXruW9996r+++/XzfffLMuuugi3Xrrrbrrrru0fPlySdm9lgQoOv0wmDlzpjZs2BB/LRaLacOGDaqoqLAxZekzbdo0FRcXJ+Sxvb1dmzdvjuexoqJChw8f1vbt2+PbbNy4UbFYTOXl5VlPc1+MMVqyZInWrFmjjRs3atq0aQnrZ86cqTPOOCMhn7t379a+ffsS8rlz586EL9D69euVn5/f6ybrFLFYTB0dHZ7K39VXX62dO3dqx44d8WXWrFlatGhR/G+v5LW7o0ePas+ePZo8ebJnrucVV1zRq7v/u+++q1AoJMk7958uq1at0qRJkzR//vz4a165ln/729/k9yeGBoFAQLFYTFKWr+UwGvt6ytNPP23y8vLM6tWrzR//+EdTU1NjCgsLE1pbO92RI0fMm2++ad58800jyfznf/6nefPNN82f//xnY8zprmGFhYXml7/8pXnrrbfMZz7zmT67hl166aVm8+bN5rXXXjNlZWWO6ub3L//yL6agoMC88sorCd39/va3v8W3+fKXv2ymTp1qNm7caLZt22YqKipMRUVFfH1XV79rrrnG7Nixw7z88stm4sSJjunqd//995tNmzaZvXv3mrfeesvcf//9xufzmf/5n/8xxrg/fwPp3ovHGG/k9Z577jGvvPKK2bt3r3n99ddNZWWlmTBhgjl48KAxxht53LJlixkxYoT5zne+Y5qamswTTzxhzjzzTPPzn/88vo0X7j/GnO7hOXXqVHPffff1WueFa1ldXW0+8pGPxLsZP/fcc2bChAnmq1/9anybbF1LApRufvKTn5ipU6eakSNHmjlz5pg33njD7iSlJBKJGEm9lurqamPM6e5h3/jGN0xRUZHJy8szV199tdm9e3fCMT788ENzyy23mDFjxpj8/HzzhS98wRw5csSG3PStr/xJMqtWrYpv8/e//93ccccd5uyzzzZnnnmm+exnP2v279+fcJxoNGquu+46M3r0aDNhwgRzzz33mJMnT2Y5N3374he/aEKhkBk5cqSZOHGiufrqq+PBiTHuz99AegYoXsjr5z73OTN58mQzcuRI85GPfMR87nOfSxgfxAt5NMaYF154wVx44YUmLy/PzJgxw9TX1yes98L9xxhjfv3rXxtJvdJujDeuZXt7u7nzzjvN1KlTzahRo8z06dPN17/+9YRu0Nm6lj5jug0PBwAA4AC0QQEAAI5DgAIAAByHAAUAADgOAQoAAHAcAhQAAOA4BCgAAMBxCFAAAIDjEKAAAADHIUABAACOQ4ACAAAchwAFAAA4DgEKAABwnP8HFXIc7vtDGDUAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plt.plot(np.asfarray(all_values[1:]), '.k')" + ] + }, + { + "cell_type": "markdown", + "id": "7e585758", + "metadata": {}, + "source": [ + "## Trainning a NN for MNIST dataset" + ] + }, + { + "cell_type": "code", + "execution_count": 34, + "id": "598dc3c4", + "metadata": {}, + "outputs": [], + "source": [ + "# number of input, hidden and output nodes\n", + "input_nodes = 784\n", + "hidden_nodes = 200\n", + "output_nodes = 10\n", + "\n", + "# learning rate\n", + "learning_rate = 0.1\n", + "\n", + "# create instance of neural network\n", + "nn1 = NeuralNetwork(input_nodes,hidden_nodes,output_nodes, learning_rate)" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "id": "b68b3583", + "metadata": {}, + "outputs": [], + "source": [ + "# epochs is the number of times the training data set is used for training\n", + "epochs = 1\n", + "\n", + "for e in range(epochs):\n", + " # go through all records in the training data set\n", + " for record in data_list:\n", + " # split the record by the ',' commas\n", + " all_values = record.split(',')\n", + " # scale and shift the inputs\n", + " inputs = (np.asfarray(all_values[1:]) / 255.0 * 0.99) + 0.01\n", + " # create the target output values (all 0.01, except the desired label which is 0.99)\n", + " targets = np.zeros(output_nodes) + 0.01\n", + " # all_values[0] is the target label for this record\n", + " targets[int(all_values[0])] = 0.99\n", + " nn1.backpropagation(inputs, targets)\n", + " pass\n", + " pass" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "id": "5af423fd", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "'4'" + ] + }, + "execution_count": 26, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "all_values[0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "22ed5d0c", + "metadata": {}, + "outputs": [], + "source": [ + "targets" + ] + }, + { + "cell_type": "markdown", + "id": "c26457cb", + "metadata": {}, + "source": [ + "## Testing the trinning NN" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "9df152e5", + "metadata": {}, + "outputs": [], + "source": [ + "# load the mnist test data CSV file into a list\n", + "test_data_file = open(\"mnist_test.csv\", 'r')\n", + "test_data_list = test_data_file.readlines()\n", + "test_data_file.close()" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "id": "98ec6bf3", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 30, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbZ0lEQVR4nO3df2xV9f3H8dfl1wWlvV0t7e0dBQuobEK7jEHXoQxHQ9slRpAY/LWBcTCxOIE5XaeCbmadmDgj6SBLJswo/koEJttwWmwbtxYDQgj70VBSpQRalIV7S4FS6ef7B+Hue6X8OJd7+27L85GchN57Pr1vD2c8d7iXU59zzgkAgB42wHoAAMCViQABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATg6wH+LKuri4dPHhQKSkp8vl81uMAADxyzqmtrU2hUEgDBpz/OqfXBejgwYPKycmxHgMAcJmam5s1cuTI8z7f6wKUkpIi6czgqampxtMAALyKRCLKycmJ/nl+PkkLUGVlpZ577jm1tLQoPz9fq1at0pQpUy667uxfu6WmphIgAOjDLvY2SlI+hPDGG29o2bJlWrFihT7++GPl5+eruLhYhw8fTsbLAQD6oKQE6Pnnn9eCBQt033336etf/7rWrFmjq666Si+99FIyXg4A0AclPECnTp3Sjh07VFRU9L8XGTBARUVFqqurO2f/jo4ORSKRmA0A0P8lPECff/65Tp8+raysrJjHs7Ky1NLScs7+FRUVCgQC0Y1PwAHAlcH8H6KWl5crHA5Ht+bmZuuRAAA9IOGfgsvIyNDAgQPV2toa83hra6uCweA5+/v9fvn9/kSPAQDo5RJ+BTRkyBBNmjRJVVVV0ce6urpUVVWlwsLCRL8cAKCPSsq/A1q2bJnmzZunb33rW5oyZYpeeOEFtbe367777kvGywEA+qCkBGju3Ln67LPPtHz5crW0tOgb3/iGtmzZcs4HEwAAVy6fc85ZD/H/RSIRBQIBhcNh7oQAAH3Qpf45bv4pOADAlYkAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwMch6ACAZOjs741p34sQJz2uGDx/uec2f//xnz2sWL17sec2BAwc8r5GkF1980fOahQsXel4zePBgz2vQf3AFBAAwQYAAACYSHqCnnnpKPp8vZhs/fnyiXwYA0Mcl5T2gG2+8Ue+///7/XmQQbzUBAGIlpQyDBg1SMBhMxrcGAPQTSXkPaO/evQqFQhozZozuuece7d+//7z7dnR0KBKJxGwAgP4v4QEqKCjQunXrtGXLFq1evVpNTU26+eab1dbW1u3+FRUVCgQC0S0nJyfRIwEAeqGEB6i0tFR33HGH8vLyVFxcrL/85S86evSo3nzzzW73Ly8vVzgcjm7Nzc2JHgkA0Asl/dMBaWlpuv7669XY2Njt836/X36/P9ljAAB6maT/O6Bjx45p3759ys7OTvZLAQD6kIQH6JFHHlFNTY0++eQT/eMf/9Ds2bM1cOBA3XXXXYl+KQBAH5bwv4I7cOCA7rrrLh05ckQjRozQTTfdpPr6eo0YMSLRLwUA6MMSHqDXX3890d8S8Gz79u1xrbv55ps9r7nnnns8r3n11Vc9r4mHz+eLa91PfvITz2uGDh3qec29997reQ3vGfcf3AsOAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADCR9B9IB1j45z//2WOv9corr3heE89NQp944gnPa+bNm+d5jSSNGzfO85of//jHnte0tLR4XvP44497XoPeiSsgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmOBu2Oj1mpubPa95+OGHkzBJ4vzpT3/yvKa4uNjzmoEDB3peI0k///nPPa959tlnPa85ePCg5zXoP7gCAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMcDNS9Hq//vWvPa85efJkEibpXldXl+c16enpntfEe2PReCxfvtzzmpqaGs9r1qxZ43nN3LlzPa+ZNm2a5zVIPq6AAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAAT3IwUvZ7P5+uRNfEaNmyY5zVpaWmJHySB/H6/5zV5eXme19TX13te8/DDD3tes3PnTs9rkHxcAQEATBAgAIAJzwGqra3VrbfeqlAoJJ/Pp40bN8Y875zT8uXLlZ2drWHDhqmoqEh79+5N1LwAgH7Cc4Da29uVn5+vysrKbp9fuXKlXnzxRa1Zs0bbtm3T1VdfreLi4h79AWEAgN7P84cQSktLVVpa2u1zzjm98MILeuKJJ3TbbbdJkl5++WVlZWVp48aNuvPOOy9vWgBAv5HQ94CamprU0tKioqKi6GOBQEAFBQWqq6vrdk1HR4cikUjMBgDo/xIaoJaWFklSVlZWzONZWVnR576soqJCgUAguuXk5CRyJABAL2X+Kbjy8nKFw+Ho1tzcbD0SAKAHJDRAwWBQktTa2hrzeGtra/S5L/P7/UpNTY3ZAAD9X0IDlJubq2AwqKqqquhjkUhE27ZtU2FhYSJfCgDQx3n+FNyxY8fU2NgY/bqpqUm7du1Senq6Ro0apSVLluiZZ57Rddddp9zcXD355JMKhUKaNWtWIucGAPRxngO0fft23XLLLdGvly1bJkmaN2+e1q1bp0cffVTt7e1auHChjh49qptuuklbtmzR0KFDEzc1AKDP8znnnPUQ/18kElEgEFA4HOb9IEiSHnzwQc9rfv/73ydhku5t3rzZ85qSkpIkTGJr27ZtntdMnTo1CZOc64svvuiR18EZl/rnuPmn4AAAVyYCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCY8PzjGIDL0dbW5nnNK6+8koRJEqc/3tk6HiNGjLAeAX0MV0AAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAluRooedeLECc9r2tvbkzAJegPnnPUIMMQVEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABggpuRoke99NJLntf4fL4kTILegN/bKxtXQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACW5Gih61evXqHnmda6+9Nq51xcXFiR0EwHlxBQQAMEGAAAAmPAeotrZWt956q0KhkHw+nzZu3Bjz/Pz58+Xz+WK2kpKSRM0LAOgnPAeovb1d+fn5qqysPO8+JSUlOnToUHR77bXXLmtIAED/4/lDCKWlpSotLb3gPn6/X8FgMO6hAAD9X1LeA6qurlZmZqZuuOEGLVq0SEeOHDnvvh0dHYpEIjEbAKD/S3iASkpK9PLLL6uqqkrPPvusampqVFpaqtOnT3e7f0VFhQKBQHTLyclJ9EgAgF4o4f8O6M4774z+euLEicrLy9PYsWNVXV2tGTNmnLN/eXm5li1bFv06EokQIQC4AiT9Y9hjxoxRRkaGGhsbu33e7/crNTU1ZgMA9H9JD9CBAwd05MgRZWdnJ/ulAAB9iOe/gjt27FjM1UxTU5N27dql9PR0paen6+mnn9acOXMUDAa1b98+Pfrooxo3bhy3OAEAxPAcoO3bt+uWW26Jfn32/Zt58+Zp9erV2r17t/74xz/q6NGjCoVCmjlzpn71q1/J7/cnbmoAQJ/nOUDTp0+Xc+68z7/77ruXNRD6twMHDnhe4/P5PK+J94r7Qv/AGkBicS84AIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmEj4j+QGLuRCd1JH31ZbW+t5TTznw+zZsz2vQe/EFRAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIKbkaJH+Xy+Hllzxx13eF6Dy/PRRx95XhPP7+0zzzzjeQ16J66AAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAAT3IwU/dLo0aOtR+jTOjs7Pa/55JNPEj9IN0KhUI+8DpKPKyAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQ3I0W/tGbNmrjWPfvsswmepG/60Y9+5HnN3/72N89rfvCDH3heM3z4cM9r0DtxBQQAMEGAAAAmPAWooqJCkydPVkpKijIzMzVr1iw1NDTE7HPy5EmVlZXpmmuu0fDhwzVnzhy1trYmdGgAQN/nKUA1NTUqKytTfX293nvvPXV2dmrmzJlqb2+P7rN06VK98847euutt1RTU6ODBw/q9ttvT/jgAIC+zdOHELZs2RLz9bp165SZmakdO3Zo2rRpCofD+sMf/qD169fre9/7niRp7dq1+trXvqb6+np9+9vfTtzkAIA+7bLeAwqHw5Kk9PR0SdKOHTvU2dmpoqKi6D7jx4/XqFGjVFdX1+336OjoUCQSidkAAP1f3AHq6urSkiVLNHXqVE2YMEGS1NLSoiFDhigtLS1m36ysLLW0tHT7fSoqKhQIBKJbTk5OvCMBAPqQuANUVlamPXv26PXXX7+sAcrLyxUOh6Nbc3PzZX0/AEDfENc/RF28eLE2b96s2tpajRw5Mvp4MBjUqVOndPTo0ZiroNbWVgWDwW6/l9/vl9/vj2cMAEAf5ukKyDmnxYsXa8OGDdq6datyc3Njnp80aZIGDx6sqqqq6GMNDQ3av3+/CgsLEzMxAKBf8HQFVFZWpvXr12vTpk1KSUmJvq8TCAQ0bNgwBQIB3X///Vq2bJnS09OVmpqqhx56SIWFhXwCDgAQw1OAVq9eLUmaPn16zONr167V/PnzJUm//e1vNWDAAM2ZM0cdHR0qLi7W7373u4QMCwDoPzwFyDl30X2GDh2qyspKVVZWxj0U+q97773X85pXXnnF85p4775x8uRJz2uGDh3qec2JEyc8r/n00089r3n88cc9r5GkDRs2eF5TUFDgec2qVas8rxkwgDuI9Rf8TgIATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMCEz13KLa57UCQSUSAQUDgcVmpqqvU4SLD29nbPa8aNG+d5zWeffeZ5jSRNnDjR85p4ftjiu+++63lNPHfDjtfkyZM9r4nnv4n/jfdPl/rnOFdAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJbkaKXm/VqlWe1yxdujQJk3Qvnv8J+Xy+JExyrry8vLjW1dbWel4zfPjwuF4L/Q83IwUA9GoECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAluRope74svvvC8prGxMa7X+s53vuN5TTgc9rzmhz/8oec1c+bM8bymqKjI8xpJGjp0aFzrAImbkQIAejkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwMQg6wGAixk0yPtpOn78+Lhe67///W9c6wB4xxUQAMAEAQIAmPAUoIqKCk2ePFkpKSnKzMzUrFmz1NDQELPP9OnT5fP5YrYHHnggoUMDAPo+TwGqqalRWVmZ6uvr9d5776mzs1MzZ85Ue3t7zH4LFizQoUOHotvKlSsTOjQAoO/z9O7uli1bYr5et26dMjMztWPHDk2bNi36+FVXXaVgMJiYCQEA/dJlvQd09kcRp6enxzz+6quvKiMjQxMmTFB5ebmOHz9+3u/R0dGhSCQSswEA+r+4P4bd1dWlJUuWaOrUqZowYUL08bvvvlujR49WKBTS7t279dhjj6mhoUFvv/12t9+noqJCTz/9dLxjAAD6KJ9zzsWzcNGiRfrrX/+qDz/8UCNHjjzvflu3btWMGTPU2NiosWPHnvN8R0eHOjo6ol9HIhHl5OQoHA4rNTU1ntEAAIYikYgCgcBF/xyP6wpo8eLF2rx5s2pray8YH0kqKCiQpPMGyO/3y+/3xzMGAKAP8xQg55weeughbdiwQdXV1crNzb3oml27dkmSsrOz4xoQANA/eQpQWVmZ1q9fr02bNiklJUUtLS2SpEAgoGHDhmnfvn1av369vv/97+uaa67R7t27tXTpUk2bNk15eXlJ+Q8AAPRNnt4D8vl83T6+du1azZ8/X83Nzbr33nu1Z88etbe3KycnR7Nnz9YTTzxxye/nXOrfHQIAeqekvAd0sVbl5OSopqbGy7cEAFyhuBccAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMDEIOsBvsw5J0mKRCLGkwAA4nH2z++zf56fT68LUFtbmyQpJyfHeBIAwOVoa2tTIBA47/M+d7FE9bCuri4dPHhQKSkp8vl8Mc9FIhHl5OSoublZqampRhPa4zicwXE4g+NwBsfhjN5wHJxzamtrUygU0oAB53+np9ddAQ0YMEAjR4684D6pqalX9Al2FsfhDI7DGRyHMzgOZ1gfhwtd+ZzFhxAAACYIEADARJ8KkN/v14oVK+T3+61HMcVxOIPjcAbH4QyOwxl96Tj0ug8hAACuDH3qCggA0H8QIACACQIEADBBgAAAJvpMgCorK3Xttddq6NChKigo0EcffWQ9Uo976qmn5PP5Yrbx48dbj5V0tbW1uvXWWxUKheTz+bRx48aY551zWr58ubKzszVs2DAVFRVp7969NsMm0cWOw/z58885P0pKSmyGTZKKigpNnjxZKSkpyszM1KxZs9TQ0BCzz8mTJ1VWVqZrrrlGw4cP15w5c9Ta2mo0cXJcynGYPn36OefDAw88YDRx9/pEgN544w0tW7ZMK1as0Mcff6z8/HwVFxfr8OHD1qP1uBtvvFGHDh2Kbh9++KH1SEnX3t6u/Px8VVZWdvv8ypUr9eKLL2rNmjXatm2brr76ahUXF+vkyZM9PGlyXew4SFJJSUnM+fHaa6/14ITJV1NTo7KyMtXX1+u9995TZ2enZs6cqfb29ug+S5cu1TvvvKO33npLNTU1OnjwoG6//XbDqRPvUo6DJC1YsCDmfFi5cqXRxOfh+oApU6a4srKy6NenT592oVDIVVRUGE7V81asWOHy8/OtxzAlyW3YsCH6dVdXlwsGg+65556LPnb06FHn9/vda6+9ZjBhz/jycXDOuXnz5rnbbrvNZB4rhw8fdpJcTU2Nc+7M7/3gwYPdW2+9Fd3n3//+t5Pk6urqrMZMui8fB+ec++53v+sefvhhu6EuQa+/Ajp16pR27NihoqKi6GMDBgxQUVGR6urqDCezsXfvXoVCIY0ZM0b33HOP9u/fbz2SqaamJrW0tMScH4FAQAUFBVfk+VFdXa3MzEzdcMMNWrRokY4cOWI9UlKFw2FJUnp6uiRpx44d6uzsjDkfxo8fr1GjRvXr8+HLx+GsV199VRkZGZowYYLKy8t1/Phxi/HOq9fdjPTLPv/8c50+fVpZWVkxj2dlZek///mP0VQ2CgoKtG7dOt1www06dOiQnn76ad18883as2ePUlJSrMcz0dLSIkndnh9nn7tSlJSU6Pbbb1dubq727dunX/ziFyotLVVdXZ0GDhxoPV7CdXV1acmSJZo6daomTJgg6cz5MGTIEKWlpcXs25/Ph+6OgyTdfffdGj16tEKhkHbv3q3HHntMDQ0Nevvttw2njdXrA4T/KS0tjf46Ly9PBQUFGj16tN58803df//9hpOhN7jzzjujv544caLy8vI0duxYVVdXa8aMGYaTJUdZWZn27NlzRbwPeiHnOw4LFy6M/nrixInKzs7WjBkztG/fPo0dO7anx+xWr/8ruIyMDA0cOPCcT7G0trYqGAwaTdU7pKWl6frrr1djY6P1KGbOngOcH+caM2aMMjIy+uX5sXjxYm3evFkffPBBzI9vCQaDOnXqlI4ePRqzf389H853HLpTUFAgSb3qfOj1ARoyZIgmTZqkqqqq6GNdXV2qqqpSYWGh4WT2jh07pn379ik7O9t6FDO5ubkKBoMx50ckEtG2bduu+PPjwIEDOnLkSL86P5xzWrx4sTZs2KCtW7cqNzc35vlJkyZp8ODBMedDQ0OD9u/f36/Oh4sdh+7s2rVLknrX+WD9KYhL8frrrzu/3+/WrVvn/vWvf7mFCxe6tLQ019LSYj1aj/rpT3/qqqurXVNTk/v73//uioqKXEZGhjt8+LD1aEnV1tbmdu7c6Xbu3Okkueeff97t3LnTffrpp845537zm9+4tLQ0t2nTJrd792532223udzcXHfixAnjyRPrQsehra3NPfLII66urs41NTW5999/333zm9901113nTt58qT16AmzaNEiFwgEXHV1tTt06FB0O378eHSfBx54wI0aNcpt3brVbd++3RUWFrrCwkLDqRPvYsehsbHR/fKXv3Tbt293TU1NbtOmTW7MmDFu2rRpxpPH6hMBcs65VatWuVGjRrkhQ4a4KVOmuPr6euuRetzcuXNddna2GzJkiPvqV7/q5s6d6xobG63HSroPPvjASTpnmzdvnnPuzEexn3zySZeVleX8fr+bMWOGa2hosB06CS50HI4fP+5mzpzpRowY4QYPHuxGjx7tFixY0O/+T1p3//2S3Nq1a6P7nDhxwj344IPuK1/5irvqqqvc7Nmz3aFDh+yGToKLHYf9+/e7adOmufT0dOf3+924cePcz372MxcOh20H/xJ+HAMAwESvfw8IANA/ESAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAm/g+bJtPV5uFCTQAAAABJRU5ErkJggg==\n", + "text/plain": [ + "
" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "all_values= test_data_list[55].split(',')\n", + "image_array= np.asfarray(all_values[1:]).reshape((28,28))\n", + "plt.imshow(image_array, cmap='Greys',interpolation='None')" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "id": "b1f6858d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[1.],\n", + " [1.],\n", + " [1.],\n", + " [1.],\n", + " [1.],\n", + " [1.],\n", + " [1.],\n", + " [1.],\n", + " [1.],\n", + " [1.]])" + ] + }, + "execution_count": 31, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "nn1.feedforward((np.asfarray(all_values[1:])/255.0*0.99)+0.01)" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "id": "4eb2a993", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "array([[0.18726904, 0.08814464, 0.76482022, ..., 0.70586941, 0.1568063 ,\n", + " 0.4999693 ],\n", + " [0.53312775, 0.84159698, 0.45797607, ..., 0.71989791, 0.67931778,\n", + " 0.83039971],\n", + " [0.99283059, 0.35872355, 0.15924308, ..., 0.62648659, 0.97066054,\n", + " 0.03276188],\n", + " ...,\n", + " [0.6879452 , 0.26462095, 0.17300758, ..., 0.02829032, 0.16182483,\n", + " 0.45969922],\n", + " [0.17684306, 0.41909022, 0.21317277, ..., 0.34570399, 0.96639186,\n", + " 0.71681652],\n", + " [0.45144587, 0.8926533 , 0.41438135, ..., 0.75578523, 0.88247098,\n", + " 0.8760342 ]])" + ] + }, + "execution_count": 32, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "nn1.who" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "017ed6db", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.10" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/slides/ann.pdf b/slides/ann.pdf new file mode 100644 index 0000000..f90ca98 Binary files /dev/null and b/slides/ann.pdf differ diff --git a/slides/logistic-regressor.pdf b/slides/logistic-regressor.pdf new file mode 100644 index 0000000..14746d9 Binary files /dev/null and b/slides/logistic-regressor.pdf differ