machine-learning-course-uac/4-ann/main.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d11e7334",
   "metadata": {},
   "source": [
    "# ANN - Example 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2b74c408",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "W = np.array([[0.9, 0.3, 0.4],\n",
    "            [0.2, 0.8, 0.2],\n",
    "            [0.1, 0.5, 0.6]])\n",
    "I = np.array([[0.9],[0.1], [0.8]])\n",
    "\n",
    "def mySigmoid(x):\n",
    "    return 1/(1+np.exp(-x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a24d741f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.16],\n",
       "       [0.42],\n",
       "       [0.62]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Xih = np.dot(W,I)\n",
    "Xih"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e977d118",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.76133271],\n",
       "       [0.60348325],\n",
       "       [0.65021855]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Oh = mySigmoid(Xih)\n",
    "Oh"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9801c7d7",
   "metadata": {},
   "source": [
    "# Creating Class and Methods\n",
    "A class defines the structure, data and methods that an object will have. There is possible to have public and private variables to operate in the methods.\n",
    "\n",
    "\n",
    "- Argument\n",
    "- Initialitation\n",
    "- Methods\n",
    "- Destroy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ec1cead5",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Dog:\n",
    "    # init method\n",
    "    def __init__(self, dogName, dogAge):\n",
    "        self.name=dogName\n",
    "        self.age=dogAge\n",
    "    \n",
    "    # status method\n",
    "    def status(self):\n",
    "        print(\"The Dogs name is: \", self.name)\n",
    "        print(\"The Dogs age is: \", self.age)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5a484af0",
   "metadata": {},
   "outputs": [],
   "source": [
    "perro1=Dog('cuadrado', 9)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "62857352",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The Dogs name is:  cuadrado\n",
      "The Dogs age is:  9\n"
     ]
    }
   ],
   "source": [
    "perro1.status()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7e848bc3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuadrado\n",
      "9\n"
     ]
    }
   ],
   "source": [
    "print(perro1.name)\n",
    "print(perro1.age)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9fb56224",
   "metadata": {},
   "source": [
    "# The Neural Notwork Class\n",
    "\n",
    "The next chunk of code defines the Neural Network's basic structure. We are going to implement and define the methods one at tima to understand them in a better way.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0a8eb25f",
   "metadata": {},
   "outputs": [],
   "source": [
    "class NeuralNetwork:\n",
    "    # init method\n",
    "    def __init__():\n",
    "        pass\n",
    "    \n",
    "    # NN computing method\n",
    "    def feedforward():\n",
    "        pass\n",
    "    \n",
    "    # NN trainning method    \n",
    "    def backpropagation():\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9253fc86",
   "metadata": {},
   "source": [
    "## Initialization or creation Method\n",
    "\n",
    "Let’s begin with the initialization. We know we need to set the number of input, hidden and output layer nodes. That defines the shape and size of the neural network. Thus, we’ll let them be set when a new neural network object is created by using the class' parameters. That way we retain the choice to create new neural networks of different sizes with simple methods.\n",
    "\n",
    "A good programmers, computer scientists and mathematicians, try to create more general code rather than specific code. It is a good habit, because it forces us to think about solving problems in a deeper and more general way. This means that our code can be used in more general scenarios.\n",
    "\n",
    "Then, ket us see how our code should look like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b4431606",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "class NeuralNetwork:\n",
    "    # init method\n",
    "    def __init__(self, inputN,hiddenN, outputN, lr):\n",
    "        # creates a NN with three layers (input, hidden, output)\n",
    "        # inputN - Number of input nodes\n",
    "        # hiddenN - Number of hidden nodes\n",
    "        self.inN=inputN\n",
    "        self.hiN=hiddenN\n",
    "        self.outN=outputN\n",
    "        self.lr=lr\n",
    "        \n",
    "        #weight W11 W21 W31\n",
    "            #   W12 W22 W32\n",
    "            # .....\n",
    "        self.wih=np.random.rand(self.hiN, self.inN)\n",
    "        self.who=np.random.rand(self.outN,self.hiN)\n",
    "        pass\n",
    "    \n",
    "    # NN computing method\n",
    "    def feedforward():\n",
    "        pass\n",
    "    \n",
    "    # NN trainning method    \n",
    "    def backpropagation():\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "d6eb9df0",
   "metadata": {},
   "outputs": [],
   "source": [
    "myNN=NeuralNetwork(3,3,3,0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "b874c4df",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.86760733, 0.61494357, 0.16001254],\n",
       "       [0.86192173, 0.71746821, 0.90211204],\n",
       "       [0.44345426, 0.66751851, 0.33738905]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "myNN.wih"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4975a68b",
   "metadata": {},
   "source": [
    "At this point were are only creating an object, but the myNN instance can't do any useful yet. Also, this is a good technique to start coding somethig, by keeping it small at the begining (make commits), and then grow the methods.\n",
    "\n",
    "Next, we should add more code to allow our NN class finish its initialization by creating the weight matrixes."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "544d454f",
   "metadata": {},
   "source": [
    "## Feedfordward method and weights initialization\n",
    "\n",
    "So the next step is to create the network of nodes and links. The most important part of the network is the link weights. They’re used to calculate the signal being fed forward, the error as it’s propagated backwards, and it is the link weights themselves that are refined in an attempt to to improve the network.\n",
    "\n",
    "For the basic NN, the weight matrix consist of:\n",
    "\n",
    "- A matrix that links the input and hidden layers, $Wih$, of size hidden nodes by input nodes ($hn×in$)\n",
    "- and another matrix for the links between the hidden and output layers, $Who$, of size $on×hn$ (output nodes by hidden nodes)\n",
    "\n",
    "$$X_h=W_{ih}I$$\n",
    "$$O_h=\\sigma{X_h}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "ce35cb31",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "class NeuralNetwork:\n",
    "    # init method\n",
    "    def __init__(self, inputN,hiddenN, outputN, lr):\n",
    "        # creates a NN with three layers (input, hidden, output)\n",
    "        # inputN - Number of input nodes\n",
    "        # hiddenN - Number of hidden nodes\n",
    "        self.inN=inputN\n",
    "        self.hiN=hiddenN\n",
    "        self.outN=outputN\n",
    "        self.lr=lr\n",
    "        \n",
    "        #weight W11 W21 W31\n",
    "            #   W12 W22 W32\n",
    "            # .....\n",
    "        np.random.seed(42)    \n",
    "        self.wih=np.random.rand(self.hiN, self.inN)\n",
    "        self.who=np.random.rand(self.outN,self.hiN)\n",
    "        pass\n",
    "    \n",
    "    # NN computing method\n",
    "    def feedforward(self, inputList):\n",
    "        # computing hidden output\n",
    "        inputs = np.array(inputList, ndmin=2).T\n",
    "        self.Xh = np.dot(self.wih, inputs)\n",
    "        self.af = lambda x:1/(1+np.exp(-x))\n",
    "        self.Oh = self.af(self.Xh)\n",
    "        \n",
    "        # computing output \n",
    "        self.Xo = np.dot(self.who, self.Oh)\n",
    "        self.Oo = self.af(self.Xo)\n",
    "        return self.Oo\n",
    "    \n",
    "    # NN trainning method    \n",
    "    def backpropagation():\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "c267a481",
   "metadata": {},
   "outputs": [],
   "source": [
    "myNN=NeuralNetwork(3,5,3,0.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c07d6dc",
   "metadata": {},
   "source": [
    "At this point we can review the variables or class attributes by calling them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "eff1bde8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.37454012, 0.95071431, 0.73199394],\n",
       "       [0.59865848, 0.15601864, 0.15599452],\n",
       "       [0.05808361, 0.86617615, 0.60111501],\n",
       "       [0.70807258, 0.02058449, 0.96990985],\n",
       "       [0.83244264, 0.21233911, 0.18182497]])"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "myNN.wih"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "159d06d2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.72922423],\n",
       "       [0.74638741],\n",
       "       [0.77466214]])"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "myNN.feedforward([0.3, 0.2, 0.1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "379439b8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.59283661],\n",
       "       [0.55635965],\n",
       "       [0.56236646],\n",
       "       [0.57774658],\n",
       "       [0.57697877]])"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "myNN.Oh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "4c351ffd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.37570429],\n",
       "       [0.22640073],\n",
       "       [0.25077181],\n",
       "       [0.31352966],\n",
       "       [0.31038311]])"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "myNN.Xh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "e95a288d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.72644667, 0.6712605 , 0.63947874])"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "myNN.Oo"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15769565",
   "metadata": {},
   "source": [
    "## The backpropagation and trainning method\n",
    "\n",
    "$$ \\frac{\\partial E}{\\partial w_{jk}}= -e_j\\cdot \\sigma\\left(\\sum_i w_{ij} o_i\\right) \\left(1-\\sigma\\left(\\sum_i w_{ij} o_i\\right) \\right) o_i $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3006bfec",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "class NeuralNetwork:\n",
    "    # init method\n",
    "    def __init__(self, inputN,hiddenN, outputN, lr):\n",
    "        # creates a NN with three layers (input, hidden, output)\n",
    "        # inputN - Number of input nodes\n",
    "        # hiddenN - Number of hidden nodes\n",
    "        self.inN=inputN\n",
    "        self.hiN=hiddenN\n",
    "        self.outN=outputN\n",
    "        self.lr=lr\n",
    "        \n",
    "        #weight W11 W21 W31\n",
    "            #   W12 W22 W32\n",
    "            # .....\n",
    "        np.random.seed(40)    \n",
    "        self.wih=np.random.rand(self.hiN, self.inN)-0.5\n",
    "        self.who=np.random.rand(self.outN,self.hiN)-0.5\n",
    "        pass\n",
    "    \n",
    "    # NN computing method\n",
    "    def feedforward(self, inputList):\n",
    "        # computing hidden output\n",
    "        inputs = np.array(inputList, ndmin=2).T\n",
    "        self.Xh = np.dot(self.wih, inputs)\n",
    "        self.af = lambda x:1/(1+np.exp(-x))\n",
    "        self.Oh = self.af(self.Xh)\n",
    "        \n",
    "        # computing output \n",
    "        self.Xo = np.dot(self.who, self.Oh)\n",
    "        self.Oo = self.af(self.Xo)\n",
    "        return self.Oo\n",
    "    \n",
    "    # NN trainning method    \n",
    "    def backpropagation(self, inputList, targetList):\n",
    "        # data\n",
    "        lr = self.lr \n",
    "        inputs = np.array(inputList, ndmin=2).T\n",
    "        target = np.array(targetList, ndmin=2).T\n",
    "        \n",
    "        #computting hidden layer\n",
    "        Xh = np.dot(self.wih, inputs)\n",
    "        af = lambda x:1/(1+np.exp(-x))\n",
    "        Oh = af(Xh)\n",
    "        \n",
    "        # computing output \n",
    "        Xo = np.dot(self.who, Oh)\n",
    "        Oo = af(Xo)\n",
    "        \n",
    "        # Output error\n",
    "        oe = target-Oo\n",
    "        # E propagation\n",
    "        hiddenE = np.dot(self.who.T, oe)\n",
    "        \n",
    "        # updating weights\n",
    "        self.who+=lr*np.dot(oe*Oo*(1-Oo), Oh.T) \n",
    "        self.wih+=lr*np.dot(hiddenE*Oh*(1-Oh), inputs.T) \n",
    "        return self.wih, self.who"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05883c85",
   "metadata": {},
   "source": [
    "## Exam"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a97b0ca8",
   "metadata": {},
   "outputs": [],
   "source": [
    "NN3 = NeuralNetwork(3,3,3,0.15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "0a101372",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-0.09231297, -0.44463396,  0.28853488],\n",
       "       [-0.21269482, -0.04964941, -0.19608769],\n",
       "       [ 0.02639952,  0.12381221,  0.27677546]])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "NN3.wih"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "8c4ed22a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.18624165,  0.48093886,  0.10081609],\n",
       "       [ 0.31396852,  0.20864515, -0.47246532],\n",
       "       [ 0.40426722, -0.05009515, -0.38107535]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "NN3.who"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b1d8aa2",
   "metadata": {},
   "source": [
    "## First Feed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a74e4922",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.58611654],\n",
       "       [0.48786288],\n",
       "       [0.4846667 ]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "NN3.feedforward([0.43, 0.88, 0.95])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d55ae872",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[-0.09474099, -0.44960292,  0.28317066],\n",
       "        [-0.21423683, -0.05280516, -0.19949447],\n",
       "        [ 0.02659345,  0.12420909,  0.2772039 ]]),\n",
       " array([[ 0.18060506,  0.47579807,  0.09354318],\n",
       "        [ 0.3176326 ,  0.21198694, -0.46773754],\n",
       "        [ 0.3976255 , -0.05615266, -0.38964518]]))"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "NN3.backpropagation([0.43, 0.88, 0.95], [0.25, 0.7, 0.1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86153af2",
   "metadata": {},
   "source": [
    "## Second feed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "6b7af517",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.4262625 ],\n",
       "       [0.50988177],\n",
       "       [0.52984418]])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "NN3.feedforward([0.47, 0.07, 0.64])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "388f333c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[ 0.34418188,  0.05427654, -0.04100328],\n",
       "        [-0.14923924,  0.08653238,  0.03279852],\n",
       "        [ 0.32666948, -0.23622231, -0.02194774],\n",
       "        [-0.15480446, -0.36651948, -0.24284465]]),\n",
       " array([[-0.44498126, -0.09860092, -0.12175549,  0.0634736 ],\n",
       "        [ 0.39189539, -0.33801846, -0.09787605,  0.14729404],\n",
       "        [ 0.04218888,  0.0429173 , -0.00094672,  0.08642189]]))"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "NN3.backpropagation([0.47, 0.07, 0.64], [0.17, 0.67, 0.14])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c146df50",
   "metadata": {},
   "source": [
    "## Third feed"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2510fdf",
   "metadata": {},
   "source": [
    "## Fourth feed"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2599e87a",
   "metadata": {},
   "source": [
    "## Fifth feed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8afe16a9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b46b2eff",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6401ecc0",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "9b35dffb",
   "metadata": {},
   "source": [
    "# MNIST Dataset\n",
    "\n",
    "Reading the complete file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b430564a",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_file= open(\"mnist_train.csv\", 'r')\n",
    "data_list= data_file.readlines()\n",
    "data_file.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcacef2d",
   "metadata": {},
   "source": [
    "## Interpreting one intrance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b7e043f7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0xffff81ac1910>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAam0lEQVR4nO3dfUyV9/3/8dfxhuMdHIcIByY61FZdVcycMmL1awdDWea8+0Nrl2hnNDpsqs61cWu1uiVsNukaDdMs2bRN6s1MqqZmM7NYMN3QRapxpi0Tw6oOwdUNDmJFI5/fH8az31GsXngObw4+H8mVyDnXh+vda1d47vIcDz7nnBMAAB2sm/UAAIDHEwECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmelgPcLfW1lbV1tYqMTFRPp/PehwAgEfOOTU1NSkjI0Pdut3/PqfTBai2tlaZmZnWYwAAHtGFCxc0aNCg+z7f6QKUmJgo6fbgSUlJxtMAALwKhULKzMwM/zy/n5gFqKSkRK+//rrq6uqUnZ2tLVu2aOLEiQ9cd+ev3ZKSkggQAMSxB72MEpM3IezZs0erV6/W+vXr9dFHHyk7O1vTpk3T5cuXY3E4AEAcikmA3njjDS1ZskTPP/+8vv71r2vbtm3q06ePfv/738ficACAOBT1AN24cUOVlZXKz8//30G6dVN+fr4qKiru2b+lpUWhUChiAwB0fVEP0Oeff65bt24pLS0t4vG0tDTV1dXds39xcbECgUB44x1wAPB4MP+HqGvXrlVjY2N4u3DhgvVIAIAOEPV3waWkpKh79+6qr6+PeLy+vl7BYPCe/f1+v/x+f7THAAB0clG/A0pISND48eNVWloafqy1tVWlpaXKzc2N9uEAAHEqJv8OaPXq1Vq4cKG++c1vauLEiXrzzTfV3Nys559/PhaHAwDEoZgEaN68efr3v/+tdevWqa6uTuPGjdOhQ4fueWMCAODx5XPOOesh/n+hUEiBQECNjY18EgIAxKGH/Tlu/i44AMDjiQABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDRw3oAAPDik08+8bwmPz+/Xcc6deqU5zUDBw5s17EeR9wBAQBMECAAgImoB+i1116Tz+eL2EaOHBntwwAA4lxMXgN66qmn9P777//vID14qQkAECkmZejRo4eCwWAsvjUAoIuIyWtAZ8+eVUZGhoYOHarnnntO58+fv+++LS0tCoVCERsAoOuLeoBycnK0Y8cOHTp0SFu3blVNTY0mT56spqamNvcvLi5WIBAIb5mZmdEeCQDQCfmccy6WB2hoaNCQIUP0xhtvaPHixfc839LSopaWlvDXoVBImZmZamxsVFJSUixHAxCH+HdAnV8oFFIgEHjgz/GYvzugf//+evLJJ1VdXd3m836/X36/P9ZjAAA6mZj/O6CrV6/q3LlzSk9Pj/WhAABxJOoBWrNmjcrLy/XPf/5Tf/3rXzV79mx1795dzz77bLQPBQCIY1H/K7iLFy/q2Wef1ZUrVzRw4EA9/fTTOnbsGH8vCgCIEPUA7d69O9rfsks4e/as5zX//e9/Pa+ZOHGi5zVAPDl+/LjnNXl5eTGYBI+Kz4IDAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEzE/BfS4bbS0lLPaz799FPPa/gwUsST9vxC5vZ8sO8//vEPz2sQe9wBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwASfht1BNm/e7HlNQUFBDCYBOo+rV696XlNcXOx5zYsvvuh5jSQNHDiwXevwcLgDAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBM8GGkHeTWrVvWIwCdzrJlyzrkOKNGjeqQ48Ab7oAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABN8GGk71NbWel7zr3/9KwaTAPHtP//5T4cc5zvf+U6HHAfecAcEADBBgAAAJjwH6OjRo5oxY4YyMjLk8/m0f//+iOedc1q3bp3S09PVu3dv5efn6+zZs9GaFwDQRXgOUHNzs7Kzs1VSUtLm85s2bdLmzZu1bds2HT9+XH379tW0adN0/fr1Rx4WANB1eH4TQmFhoQoLC9t8zjmnN998U6+88opmzpwpSXr77beVlpam/fv3a/78+Y82LQCgy4jqa0A1NTWqq6tTfn5++LFAIKCcnBxVVFS0uaalpUWhUChiAwB0fVENUF1dnSQpLS0t4vG0tLTwc3crLi5WIBAIb5mZmdEcCQDQSZm/C27t2rVqbGwMbxcuXLAeCQDQAaIaoGAwKEmqr6+PeLy+vj783N38fr+SkpIiNgBA1xfVAGVlZSkYDKq0tDT8WCgU0vHjx5WbmxvNQwEA4pznd8FdvXpV1dXV4a9ramp06tQpJScna/DgwVq5cqV+8Ytf6IknnlBWVpZeffVVZWRkaNasWdGcGwAQ5zwH6MSJE3rmmWfCX69evVqStHDhQu3YsUMvvfSSmpubtXTpUjU0NOjpp5/WoUOH1KtXr+hNDQCIe54DNHXqVDnn7vu8z+fTxo0btXHjxkcarDP785//7HnNtWvXYjAJ0Hk0Nzd7XvP3v/89BpPca8CAAR1yHHhj/i44AMDjiQABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACY8fxo2pDNnznTIccaNG9chxwGi4Wc/+5nnNbW1tZ7XjB071vOahIQEz2sQe9wBAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAm+DDSTiwnJ8d6BHQiLS0tntdUVla261i//e1vPa/Zs2dPu47l1ebNmz2v6dWrVwwmwaPiDggAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMMGHkXZiDQ0N1iNEXW1trec1ra2tnteUl5d7XiNJNTU1ntfcuHHD85otW7Z4XnPr1i3Pa/r27et5jSQVFBR4XtOeD/y8efOm5zWjRo3yvAadE3dAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJPoy0Hfr06eN5jc/n87zm+9//vuc1I0aM8LymI1VUVHhe45zzvKZHj/Zd2v369fO8Jicnx/OaNWvWeF4zefJkz2vGjRvneY3Uvg8xzczM9LymubnZ85qBAwd6XoPOiTsgAIAJAgQAMOE5QEePHtWMGTOUkZEhn8+n/fv3Rzy/aNEi+Xy+iG369OnRmhcA0EV4DlBzc7Oys7NVUlJy332mT5+uS5cuhbddu3Y90pAAgK7H8yu1hYWFKiws/NJ9/H6/gsFgu4cCAHR9MXkNqKysTKmpqRoxYoSWL1+uK1eu3HfflpYWhUKhiA0A0PVFPUDTp0/X22+/rdLSUv3qV79SeXm5CgsL7/v77IuLixUIBMJbe97KCQCIP1H/d0Dz588P/3nMmDEaO3ashg0bprKyMuXl5d2z/9q1a7V69erw16FQiAgBwGMg5m/DHjp0qFJSUlRdXd3m836/X0lJSREbAKDri3mALl68qCtXrig9PT3WhwI
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "all_values= data_list[2].split(',')\n",
    "image_array= np.asfarray(all_values[1:]).reshape((28,28))\n",
    "plt.imshow(image_array, cmap='Greys', interpolation='None')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5011b407",
   "metadata": {},
   "source": [
    "## What the NN sees?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "1f920b2d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0xffff54c853d0>]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAGdCAYAAAA44ojeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA3hUlEQVR4nO3de3TU9Z3/8dfMIAGFJHJLSAdGMKlovdTlEqO1UswxKtJa2T3VpRraqUnXsCtqrdJubV3b0tOes23tcdFICj2nXqp7hCpVuxRGrDZyq1SwFRNkSr6VgJUlAVoCZD6/P/hlNpPrTDIz38s8H+d8z0nme5nP5/ud+X7f87n6jDFGAAAADuK3OwEAAAA9EaAAAADHIUABAACOQ4ACAAAchwAFAAA4DgEKAABwHAIUAADgOAQoAADAcUbYnYChiMViev/99zV27Fj5fD67kwMAAJJgjNGRI0dUUlIiv3/gMhJXBijvv/++pkyZYncyAADAELS0tCgYDA64jSsDlLFjx0o6ncH8/HybUwMAAJLR3t6uKVOmxJ/jA3FlgNJVrZOfn0+AAgCAyyTTPINGsgAAwHEIUAAAgOMQoAAAAMchQAEAAI5DgAIAAByHAAUAADgOAQoAAHAcAhQAAOA4BCgAAMBxCFAAAIDjEKAAAIbEsixFIhFZlmV3UobMC3nwKgIUAEDKGhoaFAqFNG/ePIVCITU0NNidpJR5IQ9e5jPGGLsTkar29nYVFBSora2NyQJhK8uy1NTUpLKyskGnDs/2/skeu7/tktl/sG0yvT4V6TxWuo7p1v0ty1IoFFIsFou/FggEFI1GU06H2/OQic+Vl6X0/DYu1NbWZiSZtrY2u5OCHLZy5Urj9/uNJOP3+83KlSsds3+yx+5vu2T2H2ybTK9PRktLi9m4caP5wQ9+MOxj9eTk65/p/Tdu3Ggk9VoikUhO5SEdn9Fck8rzmwAFGIKWlpb4jalrCQQCpqWlxfb9kz12f9tt2bJl0P0He49Mr++ejo0bN/Z53ro/PHoufr8/6XOd6vln/+TYnQa7989VqTy/aYMCDEFTU1NC0bAkdXZ2qrm52fb9kz12f9u99tprg+4/2Htker00cPsBy7JUU1PT6xhdYrGYGhsb+1yXDCdf/2zsHwwGVV9fr0AgIOl01chjjz2WUhWH2/Mw3PdHElKJfL773e+aWbNmmTFjxpiJEyeaz3zmM+add95J2Oaqq67q9WultrY2YZs///nP5vrrrzejR482EydONF/5ylfMyZMnk04HJSiwm92/vnK9BGWw9f0V33dfnnnmmaTOdbavnxv2736cSCQypFIDt+eBEpShyVgVT1VVlVm1apXZtWuX2bFjh7n++uvN1KlTzdGjR+PbXHXVVeb22283+/fvjy/dE3Lq1Clz4YUXmsrKSvPmm2+aF1980UyYMMEsW7YsIxkEMmXlypUmEAjEb0xDqT/P1P7JHru/7ZLZf7BtMrl+sPYDfT08ei5dwVx/VUSDcfL1z8b+6eD2PNj9/m6UtTYoBw8eNJLMpk2b4q9dddVV5s477+x3nxdffNH4/X7T2toaf23FihUmPz/fdHR0JPW+BChwiuH8gsz0/skeu7/tktl/sG0ytT6ZX6/dHx4+ny9hW5/PZ6qrq9PSCNep1z8b+6eD2/Ng9/u7TSrP72F1M25ublZZWZl27typCy+8UJI0d+5cvf322zLGqLi4WAsWLNA3vvENnXnmmZKkBx54QM8//7x27NgRP87evXs1ffp0/f73v9ell17a6306OjrU0dER/7+9vV1TpkyhmzGQwxoaGlRbW6vOzs54+4FwOJywjWVZam5u1llnnaXLLrus3zYp0tC7yQJIXirdjEcM9U1isZiWLl2qK664Ih6cSNI///M/KxQKqaSkRG+99Zbuu+8+7d69W88995wkqbW1VUVFRQnH6vq/tbW1z/davny5HnzwwaEmFYBDDWcMiXA4rKqqKjU3N6u0tLTP/YPBoILBoCKRyIDBifR/DRwJUABnGHKAUldXp127dum1115LeL2mpib+90UXXaTJkyfr6quv1p49e3TuuecO6b2WLVumu+++O/5/VwkKAPdqaGiI97Tx+/2qr6/vVQIymK4AZDBlZWXy+/2DlqCUlpam9P4AMmdI3YyXLFmidevWKRKJDHpzKC8vl6R416vi4mIdOHAgYZuu/4uLi/s8Rl5envLz8xMWAO7VsxtwLBZTbW1txuZD6atLaXV19bC6yQLIrJQCFGOMlixZojVr1mjjxo2aNm3aoPt0tTWZPHmyJKmiokI7d+7UwYMH49usX79e+fn5uuCCC1JJDgCXsmMMiXA4rGg0qkgkomg0qtWrVyf8n2rpDYDMSqmR7B133KEnn3xSv/zlL3XeeefFXy8oKNDo0aO1Z88ePfnkk7r++us1fvx4vfXWW7rrrrsUDAa1adMmSadvQh//+MdVUlKi73//+2ptbdWtt96qL33pS/rud7+bVDqYiwe5wMtzfKRzLhcA7pGxuXjUz3gCq1atMsYYs2/fPvPJT37SjBs3zuTl5ZnS0lJz77339upOFI1GzXXXXWdGjx5tJkyYYO655x4GagO6yYU5PhhDAsg9WetmbBdKUOBluVS60NUNuL9eOAPt59XSJcDLUnl+MxcP4DC5NMdHMBjU3LlzUwoyBpqDB4B3EKAADtPVJbY7usCelu3ePwDsQ4ACOEw6Zor1qv5Kl4YzMzEAZxryQG0AMieZUVJz0bZt2/p8/eabb1Z7e/ugXYVpuwK4B41kAbhCX42HuxusIXE6Rq4FMDw0kgXgOX1V73Q3UENi2q4A7kOAAsAV+mo83N1ADYlzqWcU4BUEKABcoWfjYZ/PFw9Y+mtIbFmWIpGIxowZQ88owGVoJAvANXo2HpbUb0Pinm1Obr31Vv385z9XZ2cnPaMAF6CRLADP6W803sbGRh07doyeUYBNUnl+U4ICwHP6a3Ny7NgxzZ07155EAUgJbVAAuFJX+5K+euIwGi/gfgQoAFxnsPl4kh2Nd6AgB4C9aIMCwFVSme15oNmSGbgNyD4GaoNr8AsWqUplTJP+Zktm4DbA+QhQYJvBiumReW4MENPRvoSB2wDnI0CBLfgFaz+3BojpmO2ZRrSA8xGgwBb8grWX2wPEcDisaDSqSCSiaDSactuRdAQ5ADKLcVBgi65fsD0bOvILNjsGChDd8pAOBoPDSmvPUWndkm8gV1CCAlvwC9ZeVHGc1l8jWgD2I0CBbYZbTI+hI0AE4HSMgwLksIHGCQGAdGMuHgBJGW47DgDIFKp4AACA4xCgAAAAxyFAAQAAjkOAAgAAHIcABcgQN85zAwBOQYACZIBb57kBAKdgHBQgzSzLUigU6jWMfzQapUsvgJyWyvObEhSgH0OtomEixNxCVR6QGQQoQB+GU0XDPDe5g6o8IHOo4gF6SEcVTUNDg2pra9XZ2Rmf54a5hryFqjwgdQx1DwzDQFU0yT54wuGwqqqqmOfGw9LxOQHQPwIUoIeuKpqev4xTraJhnhtvS9fnBEDfaIMC9BAMBlVfX69AICBJ8Soago2+5WojUT4nQGbRBgXoh2VZVNEMoqGhQTU1NYrFYvL7/aqvr8+5tjZ8ToDkpfL8JkABMCQ0EgWQKsZBAZBxjPcCIJMIUAAMCeO9AMgkAhQAQ0IjUQCZRBsUAMNCI1EAyWKgNgBZw3gvADKBKh7kvFwdxwMAnIwABTmNyd4AwJlog4KcxTgeAJBdjIMCJIFxPADAuWgki5zFZG+5wbIsvfDCC9q/f78WLFig2bNn250kAEmgBAU5i3E8vK+hoUFTpkzRHXfcoYceekhz5szR4sWL7U4WgCTQBgU5j3E8vMmyLE2ZMqXPdVu2bKEkBbAB46AAKWAcD29qamrqd93rr79OgAI4HFU8ADyprKys33VXXHFFFlMCYCgIUAB4UjAY1MqVK3u9Xl1dTekJ4AK0QQHgaZZlad26dWptbdX8+fMJTgAbpfL8JkABAABZkbGB2pYvX67Zs2dr7NixmjRpkm688Ubt3r07YZvjx4+rrq5O48eP15gxY7Rw4UIdOHA
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.asfarray(all_values[1:]), '.k')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e585758",
   "metadata": {},
   "source": [
    "## Trainning a NN for MNIST dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "598dc3c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# number of input, hidden and output nodes\n",
    "input_nodes = 784\n",
    "hidden_nodes = 200\n",
    "output_nodes = 10\n",
    "\n",
    "# learning rate\n",
    "learning_rate = 0.1\n",
    "\n",
    "# create instance of neural network\n",
    "nn1 = NeuralNetwork(input_nodes,hidden_nodes,output_nodes, learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "b68b3583",
   "metadata": {},
   "outputs": [],
   "source": [
    "# epochs is the number of times the training data set is used for training\n",
    "epochs = 1\n",
    "\n",
    "for e in range(epochs):\n",
    "    # go through all records in the training data set\n",
    "    for record in data_list:\n",
    "        # split the record by the ',' commas\n",
    "        all_values = record.split(',')\n",
    "        # scale and shift the inputs\n",
    "        inputs = (np.asfarray(all_values[1:]) / 255.0 * 0.99) + 0.01\n",
    "        # create the target output values (all 0.01, except the desired label which is 0.99)\n",
    "        targets = np.zeros(output_nodes) + 0.01\n",
    "        # all_values[0] is the target label for this record\n",
    "        targets[int(all_values[0])] = 0.99\n",
    "        nn1.backpropagation(inputs, targets)\n",
    "        pass\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "5af423fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'4'"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_values[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22ed5d0c",
   "metadata": {},
   "outputs": [],
   "source": [
    "targets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c26457cb",
   "metadata": {},
   "source": [
    "## Testing the trinning NN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "9df152e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# load the mnist test data CSV file into a list\n",
    "test_data_file = open(\"mnist_test.csv\", 'r')\n",
    "test_data_list = test_data_file.readlines()\n",
    "test_data_file.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "98ec6bf3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0xffff54e71d00>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbZ0lEQVR4nO3df2xV9f3H8dfl1wWlvV0t7e0dBQuobEK7jEHXoQxHQ9slRpAY/LWBcTCxOIE5XaeCbmadmDgj6SBLJswo/koEJttwWmwbtxYDQgj70VBSpQRalIV7S4FS6ef7B+Hue6X8OJd7+27L85GchN57Pr1vD2c8d7iXU59zzgkAgB42wHoAAMCViQABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATg6wH+LKuri4dPHhQKSkp8vl81uMAADxyzqmtrU2hUEgDBpz/OqfXBejgwYPKycmxHgMAcJmam5s1cuTI8z7f6wKUkpIi6czgqampxtMAALyKRCLKycmJ/nl+PkkLUGVlpZ577jm1tLQoPz9fq1at0pQpUy667uxfu6WmphIgAOjDLvY2SlI+hPDGG29o2bJlWrFihT7++GPl5+eruLhYhw8fTsbLAQD6oKQE6Pnnn9eCBQt033336etf/7rWrFmjq666Si+99FIyXg4A0AclPECnTp3Sjh07VFRU9L8XGTBARUVFqqurO2f/jo4ORSKRmA0A0P8lPECff/65Tp8+raysrJjHs7Ky1NLScs7+FRUVCgQC0Y1PwAHAlcH8H6KWl5crHA5Ht+bmZuuRAAA9IOGfgsvIyNDAgQPV2toa83hra6uCweA5+/v9fvn9/kSPAQDo5RJ+BTRkyBBNmjRJVVVV0ce6urpUVVWlwsLCRL8cAKCPSsq/A1q2bJnmzZunb33rW5oyZYpeeOEFtbe367777kvGywEA+qCkBGju3Ln67LPPtHz5crW0tOgb3/iGtmzZcs4HEwAAVy6fc85ZD/H/RSIRBQIBhcNh7oQAAH3Qpf45bv4pOADAlYkAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwMch6ACAZOjs741p34sQJz2uGDx/uec2f//xnz2sWL17sec2BAwc8r5GkF1980fOahQsXel4zePBgz2vQf3AFBAAwQYAAACYSHqCnnnpKPp8vZhs/fnyiXwYA0Mcl5T2gG2+8Ue+///7/XmQQbzUBAGIlpQyDBg1SMBhMxrcGAPQTSXkPaO/evQqFQhozZozuuece7d+//7z7dnR0KBKJxGwAgP4v4QEqKCjQunXrtGXLFq1evVpNTU26+eab1dbW1u3+FRUVCgQC0S0nJyfRIwEAeqGEB6i0tFR33HGH8vLyVFxcrL/85S86evSo3nzzzW73Ly8vVzgcjm7Nzc2JHgkA0Asl/dMBaWlpuv7669XY2Njt836/X36/P9ljAAB6maT/O6Bjx45p3759ys7OTvZLAQD6kIQH6JFHHlFNTY0++eQT/eMf/9Ds2bM1cOBA3XXXXYl+KQBAH5bwv4I7cOCA7rrrLh05ckQjRozQTTfdpPr6eo0YMSLRLwUA6MMSHqDXX3890d8S8Gz79u1xrbv55ps9r7nnnns8r3n11Vc9r4mHz+eLa91PfvITz2uGDh3qec29997reQ3vGfcf3AsOAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADCR9B9IB1j45z//2WOv9corr3heE89NQp944gnPa+bNm+d5jSSNGzfO85of//jHnte0tLR4XvP44497XoPeiSsgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmOBu2Oj1mpubPa95+OGHkzBJ4vzpT3/yvKa4uNjzmoEDB3peI0k///nPPa959tlnPa85ePCg5zXoP7gCAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMcDNS9Hq//vWvPa85efJkEibpXldXl+c16enpntfEe2PReCxfvtzzmpqaGs9r1qxZ43nN3LlzPa+ZNm2a5zVIPq6AAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAAT3IwUvZ7P5+uRNfEaNmyY5zVpaWmJHySB/H6/5zV5eXme19TX13te8/DDD3tes3PnTs9rkHxcAQEATBAgAIAJzwGqra3VrbfeqlAoJJ/Pp40bN8Y875zT8uXLlZ2drWHDhqmoqEh79+5N1LwAgH7Cc4Da29uVn5+vysrKbp9fuXKlXnzxRa1Zs0bbtm3T1VdfreLi4h79AWEAgN7P84cQSktLVVpa2u1zzjm98MILeuKJJ3TbbbdJkl5++WVlZWVp48aNuvPOOy9vWgBAv5HQ94CamprU0tKioqKi6GOBQEAFBQWqq6vrdk1HR4cikUjMBgDo/xIaoJaWFklSVlZWzONZWVnR576soqJCgUAguuXk5CRyJABAL2X+Kbjy8nKFw+Ho1tzcbD0SAKAHJDRAwWBQktTa2hrzeGtra/S5L/P7/UpNTY3ZAAD9X0IDlJubq2AwqKqqquhjkUhE27ZtU2FhYSJfCgDQx3n+FNyxY8fU2NgY/bqpqUm7du1Senq6Ro0apSVLluiZZ57Rddddp9zcXD355JMKhUKaNWtWIucGAPRxngO0fft23XLLLdGvly1bJkmaN2+e1q1bp0cffVTt7e1auHChjh49qptuuklbtmzR0KFDEzc1AKDP8znnnPUQ/18kElEgEFA4HOb9IEiSHnzwQc9rfv/73ydhku5t3rzZ85qSkpIkTGJr27ZtntdMnTo1CZOc64svvuiR18EZl/rnuPmn4AAAVyYCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCY8PzjGIDL0dbW5nnNK6+8koRJEqc/3tk6HiNGjLAeAX0MV0AAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAluRooedeLECc9r2tvbkzAJegPnnPUIMMQVEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABggpuRoke99NJLntf4fL4kTILegN/bKxtXQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACW5Gih61evXqHnmda6+9Nq51xcXFiR0EwHlxBQQAMEGAAAAmPAeotrZWt956q0KhkHw+nzZu3Bjz/Pz58+Xz+WK2kpKSRM0LAOgnPAeovb1d+fn5qqysPO8+JSUlOnToUHR77bXXLmtIAED/4/lDCKWlpSotLb3gPn6/X8FgMO6hAAD9X1LeA6qurlZmZqZuuOEGLVq0SEeOHDnvvh0dHYpEIjEbAKD/S3iASkpK9PLLL6uqqkrPPvusampqVFpaqtOnT3e7f0VFhQKBQHTLyclJ9EgAgF4o4f8O6M4774z+euLEicrLy9PYsWNVXV2tGTNmnLN/eXm5li1bFv06EokQIQC4AiT9Y9hjxoxRRkaGGhsbu33e7/crNTU1ZgMA9H9JD9CBAwd05MgRZWdnJ/ulAAB9iOe/gjt27FjM1UxTU5N27dql9PR
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "all_values= test_data_list[55].split(',')\n",
    "image_array= np.asfarray(all_values[1:]).reshape((28,28))\n",
    "plt.imshow(image_array, cmap='Greys',interpolation='None')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "b1f6858d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.],\n",
       "       [1.],\n",
       "       [1.],\n",
       "       [1.],\n",
       "       [1.],\n",
       "       [1.],\n",
       "       [1.],\n",
       "       [1.],\n",
       "       [1.],\n",
       "       [1.]])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nn1.feedforward((np.asfarray(all_values[1:])/255.0*0.99)+0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "4eb2a993",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.18726904, 0.08814464, 0.76482022, ..., 0.70586941, 0.1568063 ,\n",
       "        0.4999693 ],\n",
       "       [0.53312775, 0.84159698, 0.45797607, ..., 0.71989791, 0.67931778,\n",
       "        0.83039971],\n",
       "       [0.99283059, 0.35872355, 0.15924308, ..., 0.62648659, 0.97066054,\n",
       "        0.03276188],\n",
       "       ...,\n",
       "       [0.6879452 , 0.26462095, 0.17300758, ..., 0.02829032, 0.16182483,\n",
       "        0.45969922],\n",
       "       [0.17684306, 0.41909022, 0.21317277, ..., 0.34570399, 0.96639186,\n",
       "        0.71681652],\n",
       "       [0.45144587, 0.8926533 , 0.41438135, ..., 0.75578523, 0.88247098,\n",
       "        0.8760342 ]])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nn1.who"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "017ed6db",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}