ANN-From-Scratch/ANN/main.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6a937ab3",
   "metadata": {},
   "source": [
    "# Development of a Modular Python Library from Scratch for Automated ROI Segmentation in Thermal Images"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "547b1c8a",
   "metadata": {},
   "source": [
    "# Module 3: Artificial Neural Network (ANN)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c29cdf01",
   "metadata": {},
   "source": [
    "Author: Sofia Samaniego Lopez\n",
    "\n",
    "Institution: Universidad Autonoma de Baja California (UABC)\n",
    "\n",
    "Advisor: Dr. Gerardo Marx Chavez Campos"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe3535d2",
   "metadata": {},
   "source": [
    "This notebook presents **Module 3** of the library's development: the implementation of an **Artificial Neural Network (ANN) from scratch**.\n",
    "\n",
    "With the objective of maintaining algorithmic transparency and bypassing commercial \"black-box\" frameworks, the entire network architecture (weight matrix initialization, feedforward propagation, and backpropagation via gradient descent) has been programmed using strictly linear algebra through **NumPy**. \n",
    "\n",
    "As a proof of concept and baseline evaluation, the model is trained and validated using the **MNIST** dataset. This demonstrates the pure mathematical algorithm's capability to classify complex patterns prior to scaling the framework for thermal image processing."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ff6f020",
   "metadata": {},
   "source": [
    "## 1. Environment Setup & Initialization\n",
    "Importing core libraries for matrix operations and data visualization. A random seed is set to ensure reproducible weight initialization across experimental runs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "371dacfd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: numpy in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (2.5.0)\n",
      "Requirement already satisfied: matplotlib in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (3.11.0)\n",
      "Requirement already satisfied: contourpy>=1.0.1 in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (from matplotlib) (1.3.3)\n",
      "Requirement already satisfied: cycler>=0.10 in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (from matplotlib) (0.12.1)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (from matplotlib) (4.63.0)\n",
      "Requirement already satisfied: kiwisolver>=1.3.1 in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (from matplotlib) (1.5.0)\n",
      "Requirement already satisfied: numpy>=1.25 in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (from matplotlib) (2.5.0)\n",
      "Requirement already satisfied: packaging>=20.0 in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (from matplotlib) (26.2)\n",
      "Requirement already satisfied: pillow>=9 in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (from matplotlib) (12.2.0)\n",
      "Requirement already satisfied: pyparsing>=3 in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (from matplotlib) (3.3.2)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (from matplotlib) (2.9.0.post0)\n",
      "Requirement already satisfied: six>=1.5 in c:\\Users\\sofia\\ANN-From-Scratch\\.venv\\Lib\\site-packages (from python-dateutil>=2.7->matplotlib) (1.17.0)\n"
     ]
    }
   ],
   "source": [
    "!pip3 install numpy\n",
    "!pip3 install matplotlib\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "np.random.seed(12)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64958384",
   "metadata": {},
   "source": [
    "## 2. Artificial Neural Network (ANN) Architecture"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8594921c",
   "metadata": {},
   "source": [
    "Neural Network's Basic Structure "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d18ac1be",
   "metadata": {},
   "outputs": [],
   "source": [
    "class ann:\n",
    "    #init\n",
    "    def __init__():\n",
    "        pass\n",
    "\n",
    "    #feedfoward\n",
    "    def feedforward():\n",
    "        pass\n",
    "\n",
    "    #backpropagation\n",
    "    def backpropagation():\n",
    "        pass\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "83a68db8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'type'>\n"
     ]
    }
   ],
   "source": [
    "MyANN = ann\n",
    "print(type(MyANN))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9680a2a",
   "metadata": {},
   "source": [
    "### 2.1 Initialization\n",
    "Defining the network structure (input, hidden, and output layers). Synaptic weight matrices ($W_{ih}$ and $W_{ho}$) are initialized using a normal distribution to break mathematical symmetry."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7285c021",
   "metadata": {},
   "outputs": [],
   "source": [
    "class ann:\n",
    "    #init\n",
    "    def __init__(self, inputNodes: int, hiddenNodes: int, outputNodes: int):\n",
    "        # Nodes\n",
    "        inN = inputNodes # Private var or parameters\n",
    "        hN = hiddenNodes\n",
    "        oN = outputNodes\n",
    "        # Weights\n",
    "        np.random.seed(12) #seed for reproducibility\n",
    "        self.wih = np.random.randn(hN, inN) #weights for input to hidden layer\n",
    "        self.who = np.random.randn(oN, hN) #weights for hidden to output layer\n",
    "        pass\n",
    "\n",
    "    #feedfoward\n",
    "    def feedforward():\n",
    "        pass\n",
    "\n",
    "    #backpropagation\n",
    "    def backpropagation():\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b5b34c6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "MyANN = ann(3, 3, 3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5c00cf15",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.47298583, -0.68142588,  0.2424395 ],\n",
       "       [-1.70073563,  0.75314283, -1.53472134],\n",
       "       [ 0.00512708, -0.12022767, -0.80698188]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MyANN.wih"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4b0f2e37",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2.87181939, -0.59782292,  0.47245699],\n",
       "       [ 1.09595612, -1.2151688 ,  1.34235637],\n",
       "       [-0.12214979,  1.01251548, -0.91386915]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MyANN.who"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3e2ed15",
   "metadata": {},
   "source": [
    "### 2.2 Feedforward (Inference)\n",
    "So the next step is to create the network of nodes and links. The most important part of the network is the link weights. They’re used to calculate the signal being fed forward, the error as it’s propagated backwards, and it is the link weights themselves that are refined in an attempt to to improve the network.\n",
    "\n",
    "For the basic NN, the weight matrix consist of:\n",
    "\n",
    "- A matrix that links the input and hidden layers, $Wih$, of size hidden nodes by input nodes ($hn×in$)\n",
    "- and another matrix for the links between the hidden and output layers, $Who$, of size $on×hn$ (output nodes by hidden nodes)\n",
    "\n",
    "$$X_h=W_{ih}I$$\n",
    "$$O_h=\\sigma(X_h)$$\n",
    "\n",
    "\n",
    "Then, \n",
    "\n",
    "$$X_o=W_{ho}O_{h}$$\n",
    "$$O_o=\\sigma(X_o)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "31b8c4dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "class ann:\n",
    "    #init\n",
    "    def __init__(self, inputNodes: int, hiddenNodes: int, outputNodes: int):\n",
    "        # Nodes\n",
    "        inN = inputNodes # Private var or parameters\n",
    "        hN = hiddenNodes\n",
    "        oN = outputNodes\n",
    "        # Weights\n",
    "        np.random.seed(12) #seed for reproducibility\n",
    "        self.wih = np.random.randn(hN, inN) #weights for input to hidden layer\n",
    "        self.who = np.random.randn(oN, hN) #weights for hidden to output layer\n",
    "        pass\n",
    "\n",
    "    #feedfoward\n",
    "    def feedforward(self, Inputs):\n",
    "        # Forward pass to hidden layer\n",
    "        inputs = np.array(Inputs, ndmin=2).T\n",
    "        Xh = np.dot(self.wih, inputs)\n",
    "        af = lambda x: 1 / (1 + np.exp(-x))\n",
    "        Oh = af(Xh)\n",
    "        # Forward pass to output layer\n",
    "        Xo = self.who @ Oh\n",
    "        Oo = af(Xo)\n",
    "        return Oo\n",
    "\n",
    "    #backpropagation\n",
    "    def backpropagation():\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6d6b0b09",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.80230104],\n",
       "       [0.65960645],\n",
       "       [0.48247944]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "MyANN = ann(3, 3, 3)\n",
    "MyANN.feedforward([0.1, 0.2, 0.3])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dd6b848",
   "metadata": {},
   "source": [
    "### 2.3 Backpropagation (Training)\n",
    "The core learning algorithm. It calculates the prediction error, propagates it backward, and dynamically updates the weight matrices using gradient descent and the chain rule."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8b04eed",
   "metadata": {},
   "source": [
    "#### Mathematical Derivation of the Cost Function and Gradient\n",
    "\n",
    "To thoroughly understand the network's learning mechanics, we must derive the gradient of the error with respect to the synaptic weights. This procedure uses the **Chain Rule** from calculus and establishes the mathematical foundation for the Gradient Descent optimization strategy used in our backpropagation algorithm.\n",
    "\n",
    "**1. The Cost Function (SSE)**\n",
    "We define the Total Error ($E$) using the Sum of Squared Errors:\n",
    "$$E = \\frac{1}{2} \\sum (T - O_o)^2$$\n",
    "Where $T$ represents the target label and $O_o$ is the predicted output. We define the output error as $e_o = (T - O_o)$.\n",
    "\n",
    "**2. The Chain Rule Application**\n",
    "To update the weight matrix $w_{ho}$ (connecting the hidden layer to the output layer), we need to determine how a change in $w_{ho}$ impacts the total error $E$. We calculate the partial derivative using the Chain Rule:\n",
    "$$\\frac{\\partial E}{\\partial w_{ho}} = \\frac{\\partial E}{\\partial O_o} \\cdot \\frac{\\partial O_o}{\\partial X_o} \\cdot \\frac{\\partial X_o}{\\partial w_{ho}}$$\n",
    "*(Note: $X_o = w_{ho} \\cdot O_h$ represents the raw signal entering the output node before activation).*\n",
    "\n",
    "**3. Solving the Partial Derivatives**\n",
    "* **Error derivative:** How the total error changes with respect to the final output.\n",
    "  $$\\frac{\\partial E}{\\partial O_o} = -(T - O_o) = -e_o$$\n",
    "* **Activation derivative:** The derivative of the Sigmoid activation function $\\sigma(X_o)$.\n",
    "  $$\\frac{\\partial O_o}{\\partial X_o} = \\sigma(X_o)(1 - \\sigma(X_o)) = O_o(1 - O_o)$$\n",
    "* **Weight derivative:** How the raw input $X_o$ changes with respect to the weight matrix $w_{ho}$. This evaluates directly to the output of the preceding hidden layer $O_h$.\n",
    "  $$\\frac{\\partial X_o}{\\partial w_{ho}} = O_h$$\n",
    "\n",
    "**4. Final Gradient Equation**\n",
    "Multiplying these individual derivatives yields the final gradient of the error for $w_{ho}$:\n",
    "$$\\frac{\\partial E}{\\partial w_{ho}}= -e_o\\cdot \\sigma \\left(w_{ho} O_h\\right) \\left(1-\\sigma\\left (w_{ho} O_h\\right) \\right) O_h$$\n",
    "\n",
    "Thus, by substituting the activated output $O_o$, we arrive at the simplified expression:\n",
    "$$\\frac{\\partial E}{\\partial w_{ho}}= -e_o\\cdot O_o \\left(1-O_o \\right) O_h$$\n",
    "\n",
    "This precise formulation dictates the weight update rule programmed in our `backpropagation` method, scaled by the learning rate ($\\eta$) to ensure stable convergence:\n",
    "$$w_{ho_{new}} = w_{ho} + \\eta \\cdot e_o \\cdot O_o(1 - O_o) \\cdot O_h^T$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "ef1098b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "class ann:\n",
    "    #init\n",
    "    def __init__(self, inputNodes: int, hiddenNodes: int, outputNodes: int):\n",
    "        # Nodes\n",
    "        inN = inputNodes # Private var or parameters\n",
    "        hN = hiddenNodes\n",
    "        oN = outputNodes\n",
    "        # Weights\n",
    "        np.random.seed(12) #seed for reproducibility\n",
    "        self.wih = np.random.randn(hN, inN) #weights for input to hidden layer\n",
    "        self.who = np.random.randn(oN, hN) #weights for hidden to output layer\n",
    "        pass\n",
    "\n",
    "    #feedfoward\n",
    "    def feedforward(self, Inputs):\n",
    "        # Oh\n",
    "        inputs = np.array(Inputs, ndmin=2).T\n",
    "        Xh = np.dot(self.wih, inputs)\n",
    "        af = lambda x: 1 / (1 + np.exp(-x))\n",
    "        Oh = af(Xh)\n",
    "        # Oo\n",
    "        Xo = self.who @ Oh\n",
    "        Oo = af(Xo)\n",
    "        return Oo\n",
    "\n",
    "    #backpropagation\n",
    "    def backpropagation(self, Inputs, Targets, Learning):\n",
    "        lr = Learning\n",
    "        inputs = np.array(Inputs, ndmin=2).T\n",
    "        targets = np.array(Targets, ndmin=2).T\n",
    "        \n",
    "        # 1. Internal feedforward\n",
    "        Xh = self.wih @ inputs\n",
    "        af = lambda x: 1 / (1 + np.exp(-x))\n",
    "        Oh = af(Xh)\n",
    "        \n",
    "        Xo = self.who @ Oh\n",
    "        Oo = af(Xo)\n",
    "        \n",
    "        # 2. Error calculation\n",
    "        Eo = targets - Oo\n",
    "        Eh = self.who.T @ Eo\n",
    "\n",
    "        # 3. Weight matrices update\n",
    "        self.who = self.who + (lr * Eo * Oo * (1-Oo) ) @ Oh.T\n",
    "        self.wih = self.wih + (lr * Eh * Oh * (1-Oh) ) @ inputs.T\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "deacbee4",
   "metadata": {},
   "outputs": [],
   "source": [
    "MyANN = ann(3, 5, 3)\n",
    "MyANN.backpropagation([0.1, 0.2, 0.3], [0.01, 0.01, 0.99], 0.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40d8674c",
   "metadata": {},
   "source": [
    "## 3. MNIST Dataset Exploration\n",
    "Loading the training dataset. To verify the geometric structure, a raw 784-pixel flat array is extracted and reshaped into a 28x28 2D matrix for visual confirmation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "bb581e90",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function TextIOWrapper.close()>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load training data\n",
    "file = open(\"mnist_train.csv\")\n",
    "list = file.readlines()\n",
    "file.close"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "ddb0a6fb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaEAAAGdCAYAAAC7EMwUAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlcelbwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAGjRJREFUeJzt3Q9MVef9x/EvgvwpCqLOuQ6cWu2MDCwTWOufDrYo/mn9M1dpt7otNc5mWZuJcyntapsuKZndxNZ1S7PNLLXraEeok62rUTb/FJmjYnU6WPyTCrULVKv8kX8C95fn+YUb7xTlXC98ufe8X8kJPefex3N4+tzz4TnnOc8N83g8HgEAQMEwjZ0CAGAQQgAANYQQAEANIQQAUEMIAQDUEEIAADWEEABADSEEAFATIUNQT0+PfPTRRzJy5EgJCwvTPhwAgANmDoTm5ma5/fbbZdiwYcEXQiaAkpKStA8DAHAL6urqJDExUSeEjh07JmfPnpWpU6fKtGnTHJU1PaDeXyAuLm6AjhAAMBCamppsR6L3XD6oIdTZ2SkrV66UAwcOyF133SWVlZXywAMPyG9+85t+X1rrfZ8JIEIIAIJTf875AQ+hLVu2SHl5uRw9etR2w06cOCHp6emSlZUlq1atCvTuAABBLOCj47Zv3y65ubne64DJycmycOFCux0AgAELoa6uLqmurpbU1FSf7Wbd3CPqS0dHh72GePUCAAh9AQ2hlpYW6e7uloSEBJ/tY8aMkUuXLvVZrqCgQOLj470LI+MAwB0CGkJRUVH2Z2tr6zXhFB0d3We5/Px8aWxs9C5mVBwAIPQFdGBCTEyMjB8/Xmpra322m/XJkyffMLx6AwwA4B4BH5hgBiGUlJTYWQ+M9vZ2KS0ttdsBALhamMfMrxBAH3zwgWRkZMicOXNk0aJFUlRUJKdPn5bDhw/be0P9YQYmmHtD5tIczwkBQHBxcg4PeE9o4sSJUlVVZWdJ2Lt3r8ydO9c+sNrfAAIAuEfAe0KBQE8IAIKXak8IAID+IoQAAGoIIQCAGkIIAKCGEAIAqCGEAABqCCEAgBpCCACghhACAKghhAAAagghAIAaQggAoIYQAgCoIYQAAGoIIQCAGkIIAKCGEAIAqCGEAABqCCEAgBpCCACghhACAKghhAAAagghAIAaQggAoIYQAgCoIYQAAGoIIQCAGkIIAKCGEAIAqCGEAABqCCEAgBpCCACghhACAKghhAAAagghAIAaQggAoIYQAgCoIYQAAGoIIQCAGkIIAKCGEAIAqCGEAABqCCEAgBpCCACghhACAKghhAAAagghAIAaQggAoIYQAgCoIYQAAGoIIQCAGkIIAKCGEAIAqInQ2zWC3ZUrV/wqFxYW5rhMd3e34zJvvvmm4zIlJSWOy2zcuFH8kZKSMih1Fx4e7rgMELQhVFNTI8ePH/fZFhkZKUuWLAn0rgAAQS7gIVRcXCyFhYWSnZ3t3RYbG0sIAQAG53Lc1KlTbRgBADDoIdTW1ia7du2S6Ohoe9179OjRA7EbAECQG5DRcWfOnJFNmzZJXl6eJCYmyubNm2/4/o6ODmlqavJZAAChL+AhlJWVJbW1tVJWViaHDx+Wbdu2yfr16+16XwoKCiQ+Pt67JCUlBfqwAABuCKE5c+ZIQkKCd/3BBx+0l+R27tzZZ5n8/HxpbGz0LnV1dYE+LACAW58TMr2bhoaGPl+PioqyCwDAXQLeE6qvr/dZN72aqqoqSU9PD/SuAABBLuA9oaVLl0pmZqakpaXJ+fPnZevWrZKcnCxr164N9K4AAEEu4D2hffv22XtA5eXlthf0/PPPS0VFhYwYMSLQuwIABLmA94TMvZ01a9bYBQCAG2EC00Hk8Xgcl/nnP//puMzFixcHZT+vvvqq+CMnJ8dxGfMsmVPm8YDB8Kc//cmvcl/+8pcdl/FnDsbm5mbHZVauXOm4zJQpU8QfERGchtyMr3IAAKghhAAAagghAIAaQggAoIYQAgCoIYQAAGoIIQCAGkIIAKCGEAIAqCGEAABqCCEAgBpCCACgJszjz6yaA6ypqcl+G6v5qu+4uDgJFf5MdLl8+fIBORbo8+ejFxYWJkNVbm6uX+XMd445NXbsWL/2haF3DqcnBABQQwgBANQQQgAANYQQAEANIQQAUEMIAQDUEEIAADWEEABADSEEAFBDCAEA1BBCAAA1hBAAQE2E3q7dZ9myZSE1YSVwtTfeeMOvCpkxY4bjMk888QSVHyLoCQEA1BBCAAA1hBAAQA0hBABQQwgBANQQQgAANYQQAEANIQQAUEMIAQDUEEIAADWEEABADSEEAFBDCAEA1DCLNnCV+fPnO66PqKgov+pw586d1L2InDhxgnpwMXpCAAA1hBAAQA0hBABQQwgBANQQQgAANYQQAEANIQQAUEMIAQDUEEIAADWEEABADSEEAFBDCAEA1DCB6SCaNm2a4zL/+c9/HJeJjY11XKawsNBxmTvuuEP8kZSUJEPVpEmTHJcJDw/3a1+nTp1yXObgwYOOy0yePNlxmXvvvVcGyzvvvOO4zIULFxyXGTNmjOMyGKI9IfPhKS4ultra2j7fc+zYMSktLZWamppbOT4AQAhzFEJVVVWSk5MjCxYskAceeED2799/zXs6Oztl2bJlkp2dLVu2bJHMzExZvXq1eDyeQB43AMBtIWS6wOvWrZOTJ0/2+R4TPOXl5XL06FEpKyuTiooKef311+W1114LxPECANwaQvPmzbO9oLCwsD7fs337dsnNzZXExES7npycLAsXLrTbAQAYsNFxXV1dUl1dLampqT7bzbq5R9SXjo4OaWpq8lkAAKEvoCHU0tIi3d3dkpCQcM2olEuXLvVZrqCgQOLj473LUB49BQAYoiEUFRVlf7a2tl4TTtHR0X2Wy8/Pl8bGRu9SV1cXyMMCALjhOaGYmBgZP378NUO3zfqNnlUw4dUbYAAA9wj4jAlmEEJJSYn09PTY9fb2dvu8kNkOAIDfPaGGhgafZ4MqKyvtZTbzlPnMmTPtto0bN0pGRoasWLFCFi1aJEVFRRIRESF5eXlOdgUAcAFHPaH6+nobKmYxIXPu3Dn73yaMek2cONE+1GqmqNm7d6/MnTvXvs6UGQCAW+oJpaSk2Ol6bsaMbjMj3gAAuJEwzxCcT8c8J2SGapuRcnFxcRIqzChBp0wdODV8+HDHZcaNG+e4DIKDeX7PKX/+iHzmmWdksPizr8E8PrdrcnAO56scAABqCCEAgBpCCACghhACAKghhAAAagghAIAaQggAoIYQAgCoIYQAAGoIIQCAGkIIAKCGEAIAqGECUwDX+PDDDx3XyoQJEwatJrOzsx2Xefvttx2X4Ruf/cMEpgCAoMDlOACAGkIIAKCGEAIAqCGEAABqCCEAgBpCCACghhACAKghhAAAagghAIAaQggAoIYQAgCoidDbNQD4p6GhwXGZ7u5uqnsIoicEAFBDCAEA1BBCAAA1hBAAQA0hBABQQwgBANQQQgAANYQQAEANIQQAUEMIAQDUEEIAADWEEABADSEEAFDDLNoAgs7cuXMdl7ntttsG5Fhwa+gJAQDUEEIAADWEEABADSEEAFBDCAEA1BBCAAA1hBAAQA0hBABQQwgBANQQQgAANYQQAEANIQQAUMMEpkCI6+rqclymo6PDcRmPxyODZfbs2YNyfGFhYY7LYBBC6NSpU/L+++9LZmamTJgwwee1mpoaOX78uM+2yMhIWbJkiT+7AgCEMEchVFVVJfn5+XL69Gm7bN++XR5++GGf9xQXF0thYaFkZ2d7t8XGxhJCAIBbC6ELFy7IunXrJCcnR4YN6/t20tSpU20YAQAQsBCaN29ev97X1tYmu3btkujoaElJSZHRo0c72Q0AwCUGZHTcmTNnZNOmTZKXlyeJiYmyefPmm94EbWpq8lkAAKEv4CGUlZUltbW1UlZWJocPH5Zt27bJ+vXr7XpfCgoKJD4+3rskJSUF+rAAAG4IoTlz5khCQoJ3/cEHH7SX5Hbu3NlnGTPYobGx0bvU1dUF+rAAAG59Tsj0bhoaGvp8PSoqyi4AAHcJeE+ovr7eZ930aszQ7vT09EDvCgDgpp6Q6c3s37/fu15ZWWlHwE2aNElmzpxpty1dutQ+xJqWlibnz5+XrVu3SnJysqxduzbwRw8AcE9PyPRyioqK7LJixQo5d+6c/W8TRr327dtn7wGVl5fbXtDzzz8vFRUVMmLEiIE4fgCAW3pCJlxu9hCqubezZs0auwAAcCNMYArcos7OTr/KHTp0aFAm1PzBD37guIy5jzuUJ/tctWqV4zLmkRGnRo0a5bjMhg0bxB8xMTHiRnyVAwBADSEEAFBDCAEA1BBCAAA1hBAAQA0hBABQQwgBANQQQgAANYQQAEANIQQAUEMIAQDUEEIAADVhHo/HI0NMU1OT/TZW81XfcXFx2ocDFzl48KDjMk899ZRf+9q7d++QniQU/rnnnnv8KldeXh4yVe7kHE5PCACghhACAKghhAAAagghAIAaQggAoIYQAgCoIYQAAGoIIQCAGkIIAKCGEAIAqCGEAABqCCEAgJoIvV0DA+vAgQOOy+Tk5Dgu097e7rgMQldFRYX2IQQVekIAADWEEABADSEEAFBDCAEA1BBCAAA1hBAAQA0hBABQQwgBANQQQgAANYQQAEANIQQAUEMIAQDUEEIAADXMoo2Q9eijjzouw4zYuFXf+973qEQH6AkBANQQQgAANYQQAEANIQQAUEMIAQDUEEIAADWEEABADSEEAFBDCAEA1BBCAAA1hBAAQA0hBABQwwSmCArvvPOO4zIffPDBgBwLrm/MmDF+Vc1bb73luMz48eMdlzl48KDjMrNmzXJcZuzYsY7LuJnjELp8+bIcOXJErly5IikpKX1W+LFjx+Ts2bMydepUmTZtWiCOFQDg5stxzz33nNx5553yxBNPyLPPPiuf+9zn5IUXXvB5T2dnpyxbtkyys7Nly5YtkpmZKatXrxaPxxPoYwcAuKknNG7cOKmpqZGRI0fa9R07dsjy5cslKytLMjIy7DYTPOXl5XL06FFJTEyUEydOSHp6un3PqlWrBua3AACEfk/IfElYbwAZpscTGRlpL8/12r59u+Tm5toAMpKTk2XhwoV2OwAAARsd9+6779rLbyZojK6uLqmurpbU1FSf95l1c4+oLx0dHdLU1OSzAABCn98hdPHiRXnkkUdk8eLFMnv2bLutpaVFuru7JSEh4ZpRM5cuXerz3yooKJD4+HjvkpSU5O9hAQBCPYRMT8VcYhs1apS8/vrr3u1RUVH2Z2trq8/7TThFR0f3+e/l5+dLY2Ojd6mrq/PnsAAAoT5Eu7m5WRYsWGAvve3Zs0fi4uK8r8XExNjx+7W1tT5lzPrkyZP7/DdNePUGGADAPRz1hEyPxgSQuQ+0e/du2xP6X6aHVFJSIj09PXa9vb1dSktL7XYAAPzuCd1///12gEFhYaGUlZV5t0+fPt0uxsaNG+1w7RUrVsiiRYukqKhIIiIiJC8vz8muAAAu4CiEPvOZz0hOTs41U6isXLnSG0ITJ06Uqqoq+eUvfyl79+6VuXPn2iDyd0oPAEDochRCVw9CuBEzus2MeAMA4EaYwBSDztxbdOrxxx93XKatrc1xGfy/GTNmOK4KM1DJH4N1lWTKlCmDsh84w1c5AADUEEIAADWEEABADSEEAFBDCAEA1BBCAAA1hBAAQA0hBABQQwgBANQQQgAANYQQAEANIQQAUMMEphh0p0+fdlzm1KlTA3IsbnC9L5+8mddee81xGb6uBf6gJwQAUEMIAQDUEEIAADWEEABADSEEAFBDCAEA1BBCAAA1hBAAQA0hBABQQwgBANQQQgAANYQQAEANE5gCQSQ2NtZxmZ/+9KeOyyQnJzsuA/iDnhAAQA0hBABQQwgBANQQQgAANYQQAEANIQQAUEMIAQDUEEIAADWEEABADSEEAFBDCAEA1BBCAAA1hBAAQA2zaGPQpaSkOC7z1FNPOS4TFhbmuMyWLVscl2lpaRF/dHR0DMrvNHz4cMdlgMFCTwgAoIYQAgCoIYQAAGoIIQCAGkIIAKCGEAIAqCGEAABqCCEAgBpCCACghhACAKghhAAAagghAICaMI/H45EhpqmpSeLj46WxsVHi4uK0DwcAMEDncMezaF++fFmOHDkiV65csbMhjx071uf1mpoaOX78uM+2yMhIWbJkidNdAQBCnKMQeu655+SVV16RSZMmSXh4uLz33nvy7LPPyoYNG7zvKS4ulsLCQsnOzvZui42NJYQAALcWQuPGjbM9nZEjR9r1HTt2yPLlyyUrK0syMjK875s6daoNIwAAAjYw4dFHH/UGkLFs2TJ7qc1cnrtaW1ub7Nq1S/bt2yeffPKJk10AAFzklkbHvfvuu9LZ2SnJyck+28+cOSObNm2SvLw8SUxMlM2bN9/0GybNjayrFwBA6PM7hC5evCiPPPKILF68WGbPnu3dbi7N1dbWSllZmRw+fFi2bdsm69evt+t9KSgosCMpepekpCR/DwsAEOpDtE1PZf78+dLT0yN79uy56RC81NRUO1DhxRdf7LMnZJar/30TRAzRBoDgM6BDtJubm2XBggXS1dXVrwAyzME0NDT0+XpUVJRdAADu4uhyXEtLiw0gcx9o9+7dMmrUqGveU19f77NeV1cnVVVVkp6efutHCwBw7+U4c0nNPBtkngO6OoCmT59uF+Puu++WzMxMSUtLk/Pnz8vWrVtl/Pjx8re//U1GjBjRr/0wYwIABC8n53BHIfSNb3zD9oL+18qVK+1imHs7r776qhw6dEhuu+02G0gPPfSQfbh1IH4BAIBLQmiwEEIAELycnMOZRRsAoIYQAgCoIYQAAGoIIQCAGkIIAKCGEAIAqCGEAABqCCEAgBpCCACghhACAKghhAAAagghAIAaQggAoIYQAgCoIYQAAGoIIQCAGkIIAKCGEAIAqCGEAABqCCEAgBpCCACghhACAKghhAAAagghAIAaQggAoCZChiCPx2N/NjU1aR8KAMCh3nN377k86EKoubnZ/kxKStI+FADALZzL4+Pjb/ieME9/omqQ9fT0yEcffSQjR46UsLAwn3Q1wVRXVydxcXHiVtQD9UB74HMxlM8PJlZMAN1+++0ybNiw4OsJmYNOTEzs83VTsW4OoV7UA/VAe+BzMVTPDzfrAfViYAIAQA0hBABQE1QhFBUVJc8884z96WbUA/VAe+BzESrnhyE5MAEA4A5B1RMCAIQWQggAoIYQAgCoGZLPCV1Pd3e3VFRUyPnz5+Wuu+6SiRMnituY3//s2bM+2z71qU/JV7/6VQl1jY2NsmfPHvn0pz8tc+bMue57zp07J++99559PmH27NkyfPhwcVM9mIcD//KXv1xTJjs7274/VJhzwbFjx+TDDz+UO+64Q6ZPn+7K9tB9k3oIlvYQFCHU0NAg8+bNsx/AKVOmyMGDB+Xpp5+W/Px8cZMXX3xRKisrJSMjw7tt2rRpIR1C5oO0fv16+fOf/2xn0rj77ruvG0K/+MUv5Ec/+pF9vba2VsLDw2X37t0yYcIEcUs9mJPuQw89JEuWLJGYmBjv9i984QtD6qRzK3bt2iXr1q2zI7/MA+3mD7MvfvGLUlJSIiNGjHBNe9jVj3oImvbgCQLf/OY3PWlpaZ7W1la7Xlpaakb0eSorKz1ukpub61m9erXHTerr6z2vvPKKp6WlxbNixQrP0qVLr3lPdXW1Jzw83POHP/zBrnd0dHhmzZrlue+++zxuqwfzuairq/OEKvPZP3nypHf9448/9nz2s5/1bNiwwVXtobSf9RAM7WHI94Ta29uluLhYXnrpJW+a33fffXLnnXfK73//e0lPTxc3+fjjj2XHjh0yduxYmTFjhp1fL5SNGzdOvvvd797wPUVFRfYvu9zcXLseGRkp3//+9+Xhhx+WTz75REaPHi3Brj/10MtcKTCflc9//vP2cxJKzGf/auZzcO+998r777/vqvZwXz/qIVjaw5AfmHDy5Enp6OiwXcirpaSkyL/+9S9xm0OHDsmvf/1rWbt2rb0v9sc//lHczrSD5ORkn8luTfswl63+/e9/i5uYeRd//vOf28tRM2fOlPvvvz+kvxLF/JFaXl7uc35wY3tov049BEt7GPIhZO4DGf/718uYMWPk0qVL4iZr1qyx17fNzcYTJ07IY489Jt/+9rflzJkz4mamjVyvfRhuaiMJCQn2Rrz5Q8XcM6iurpYjR47Ihg0bJFSZz0Bra6v88Ic/dHV7eOw69RAs7WHIh1Dv1BMtLS0+2816dHS0uIkZgGAuLfT68Y9/bP+6M6Ol3My0keu1D8NNbcRcgkpLS/OumxvWpsdcWloqoejJJ5+UN9980w7WMF8Z4Nb28GQf9RAs7WHIh9DkyZPtT9MDuJoZqtz7mltFRETYa73mPpGbmeGp12sfhtvbiJnKPxTbhxkday4x/fWvf5UvfelLrm0PT9+gHoKlPQz5EDLdaFO5V9/7MA3sH//4hyxevFjcwtwXu3jxos+2v//97/bywtVDtt1o0aJF9j5ATU2Nd9sbb7xhh6+H2knnRv773//6rJtpId96662Qax8bN260jyuYE++sWbNc2x423qQegqU9DPnRccbPfvYzeynKjA5KTU2VX/3qV/bhs69//eviFm1tbXLPPffYMf/mw3T69GnZunWrfQ5g/vz5EsrMsw+dnZ32oTzzgJ4Z/WQuS37ta1+zry9YsMD+QWJGDD3++ONy6tQp+d3vfjfkLjsMdD389re/tTenTXswl53MH27mYca3335bQoUZJfuTn/zE/n823xxq6sAYNWqUbQduaQ8v9aMegqU9BM0s2sePH5dt27bJhQsX7CgPc20zWKYqDxRzw9V8mMxfeWZI5le+8pWQDyBj9erVcvnyZZ9tsbGx9kPW68qVK7Z9mIf2zBPy3/rWt2w7cVs97N+/355szQgoMyT3O9/5TkgMSe718ssvy4EDB67Zbr7O+oUXXnBNe3i5n/UQDO0haEIIABB6hvw9IQBA6CKEAABqCCEAgBpCCACghhACAKghhAAAagghAIAaQggAoIYQAgCoIYQAAGoIIQCAGkIIACBa/g94lJIy4CerDwAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualize sample at index 120\n",
    "values = list[120].split(\",\")\n",
    "image = np.asarray(values[1:], dtype=int)\n",
    "plt.imshow(image.reshape(28,28), cmap='Grays')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "23314858",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49999"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "values [0]\n",
    "len(list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b499d390",
   "metadata": {},
   "source": [
    "## 4. Model Training\n",
    "Setting up hyperparameters. During training, pixel intensities are normalized to a $[0.01, 1.0]$ range to prevent zero-gradient issues. Target labels are formatted using an adapted One-Hot Encoding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "c4dd5448",
   "metadata": {},
   "outputs": [],
   "source": [
    "# hyperparameters\n",
    "inputNodes = 784\n",
    "hiddenNodes = 100\n",
    "outNodes = 10\n",
    "learningRate = 0.1\n",
    "MyANN = ann(inputNodes, hiddenNodes, outNodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "c4020634",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5 - Average Loss: 0.0972\n",
      "Epoch 2/5 - Average Loss: 0.0559\n",
      "Epoch 3/5 - Average Loss: 0.0461\n",
      "Epoch 4/5 - Average Loss: 0.0403\n",
      "Epoch 5/5 - Average Loss: 0.0361\n"
     ]
    }
   ],
   "source": [
    "# Iterative training loop\n",
    "epochs = 5\n",
    "for e in range(epochs):\n",
    "    total_loss = 0\n",
    "    for record in list: \n",
    "        values = record.split(\",\")\n",
    "       \n",
    "        # Input data normalization\n",
    "        data = np.asarray(values[1:], dtype=int)/255*0.99+0.01\n",
    "        index = np.asarray(values[0],dtype=int)\n",
    "       \n",
    "        # Target Vector construction\n",
    "        target = np.zeros(outNodes) + 0.01\n",
    "        target[index] = 0.99\n",
    "        \n",
    "        # Calculate loss before updating weights\n",
    "        output = MyANN.feedforward(data)\n",
    "        # Using SSE formulation: 0.5 * sum((target - output)^2)\n",
    "        loss = np.sum(0.5 * (target.reshape(-1, 1) - output)**2)\n",
    "        total_loss += loss\n",
    "        \n",
    "        # Train\n",
    "        MyANN.backpropagation(data, target, learningRate)\n",
    "        \n",
    "    average_loss = total_loss / len(list)\n",
    "    print(f\"Epoch {e+1}/{epochs} - Average Loss: {average_loss:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c75cb916",
   "metadata": {},
   "source": [
    "## 5. Validation & Inference\n",
    "Evaluating model performance using unseen test data. A new sample is normalized and processed to extract the final prediction vector, which is then visually compared to the ground truth image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "a446646f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function TextIOWrapper.close()>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load testing data\n",
    "file2 = open(\"mnist_test.csv\")\n",
    "list2 = file2.readlines()\n",
    "file2.close"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "bb895183",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Network Accuracy (Score) on Test Set: 95.27%\n"
     ]
    }
   ],
   "source": [
    "#Test Set Evaluation (Network Score)\n",
    "scorecard = []\n",
    "\n",
    "for record in list2:\n",
    "    values = record.split(\",\")\n",
    "    correct_label = int(values[0])\n",
    "    \n",
    "    # Normalize input\n",
    "    data = np.asarray(values[1:], dtype=float) / 255.0 * 0.99 + 0.01\n",
    "    \n",
    "    # Get network prediction\n",
    "    outputs = MyANN.feedforward(data)\n",
    "    # The index of the highest value corresponds to the predicted class\n",
    "    predicted_label = np.argmax(outputs)\n",
    "    \n",
    "    # Append 1 if correct, 0 if incorrect\n",
    "    if predicted_label == correct_label:\n",
    "        scorecard.append(1)\n",
    "    else:\n",
    "        scorecard.append(0)\n",
    "\n",
    "scorecard_array = np.asarray(scorecard)\n",
    "accuracy = scorecard_array.sum() / scorecard_array.size\n",
    "print(f\"Network Accuracy (Score) on Test Set: {accuracy * 100:.2f}%\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "682673f4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[3.92210113e-05],\n",
       "       [9.64025823e-01],\n",
       "       [1.22524901e-03],\n",
       "       [1.47886768e-02],\n",
       "       [1.61851002e-03],\n",
       "       [3.98689305e-03],\n",
       "       [7.74585782e-05],\n",
       "       [2.65175424e-03],\n",
       "       [4.98386616e-03],\n",
       "       [2.37478194e-03]])"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Inference on sample 500\n",
    "values = list2[700].split(\",\")\n",
    "data = np.asarray(values[1:], dtype=int)/255*0.99+0.01\n",
    "\n",
    "# Display probability vector for the 10 classes\n",
    "MyANN.feedforward(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "8d5fb4bd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaEAAAGdCAYAAAC7EMwUAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlcelbwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAGUxJREFUeJzt3Q+QF2X9B/DPAfIn4E5AjRQKKZThgkSBTKEBHRVR8Q8pWlqNjH/GyWaELGnKHJuRBn+JRuU4FTWhRsaQ4/WPhPIfkoOgEgalMsmlDojIHQgcAt/f7M7cDZce8T2/8Nzd9/Wa2Tl3v/vcLuuz+75n99nnW1EoFAoBAAl0SrFRAMgIIQCSEUIAJCOEAEhGCAGQjBACIBkhBEAyQgiAZLpEG7Rv3754/fXXo3fv3lFRUZF6dwAoQjYGwrZt2+LYY4+NTp06tb8QygJo4MCBqXcDgA+gtrY2BgwYkCaEVq9eHa+++moMGTIkhg4dWlTZrAXU+A+orKw8RHsIwKFQX1+fNyQar+WHNYR2794dl112WTz55JNx0kknxYoVK+LSSy+Nn/70pwd9a61xvSyAhBBA+3Qw1/ySh9Ddd98dy5YtixdeeCFvhr344osxatSoGD9+fFx11VWl3hwA7VjJe8fNnz8/pk6d2nQfsLq6Os4999x8OQAcshDas2dPrF27NkaMGNFseTafPSNqSUNDQ34Pcf8JgI6vpCG0ffv22Lt3b/Tp06fZ8n79+sXWrVtbLDdr1qyoqqpqmvSMAygPJQ2hbt265T937NjxnnDq3r17i+VmzpwZdXV1TVPWKw6Ajq+kHRN69OgR/fv3jw0bNjRbns0PHjz4gOHVGGAAlI+Sd0zIOiEsWrQoH/Ugs2vXrqipqcmXA8D+KgrZ+Aol9O9//ztGjx4dY8eOjUmTJsWCBQvilVdeiZUrV+bPhg5G1jEhezaU3ZrznhBA+1LMNbzkLaFBgwbFqlWr8lESHnvssRg3blz+wurBBhAA5aPkLaFS0BICaL+StoQA4GAJIQCSEUIAJCOEAEhGCAGQjBACIBkhBEAyQgiAZIQQAMkIIQCSEUIAJCOEAEhGCAGQjBACIBkhBEAyQgiAZIQQAMkIIQCSEUIAJCOEAEhGCAGQjBACIBkhBEAyQgiAZIQQAMkIIQCSEUIAJCOEAEhGCAGQjBACIBkhBEAyQgiAZIQQAMkIIQCSEUIAJCOEAEhGCAGQjBACIBkhBEAyQgiAZIQQAMkIIQCSEUIAJCOEAEhGCAGQjBACIBkhBEAyQgiAZIQQAMkIIQCSEUIAJCOEAEhGCAGQTJd0m4aDN3v27KIP1ze+8Y2iy1RUVBRd5uqrr47WuPfee4suc8QRR7RqW1A2IbRu3bpYs2ZNs2Vdu3aNyZMnl3pTALRzJQ+hhQsXxpw5c2LChAlNy3r27CmEADg8t+OGDBmShxEAHPYQ2rlzZyxevDi6d+8ew4cPj759+x6KzQDQzh2S3nHr16/PHyRPnz49BgwYEHfdddcB129oaIj6+vpmEwAdX8lDaPz48bFhw4ZYunRprFy5MubNmxczZszI51sya9asqKqqapoGDhxY6t0CoBxCaOzYsdGnT5+m+csvvzy/JffII4+0WGbmzJlRV1fXNNXW1pZ6twAo1/eEstbNpk2bWvy8W7du+QRAeSl5S2jjxo3N5rNWzapVq2LUqFGl3hQA7VzJW0IXXnhhjBkzJkaOHBmbN2+OuXPnRnV1dVx33XWl3hQA7VzJW0KPP/54/gxo2bJleSvojjvuiOXLl0evXr1KvSkA2rmSt4SyZzvXXHNNPgHAgRjAlA6rU6fDM0j8z3/+81aVu+eee4ouYwBTOhpf5QBAMkIIgGSEEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMhWFQqEQbUx9fX1UVVVFXV1dVFZWpt4d2oB9+/YVXeboo48uuszWrVvjcHnmmWeKLjNq1KhDsi+Q6hquJQRAMkIIgGSEEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACTTJd2m4eB16lT830uzZ88uusy1114bh8udd95ZdJn58+cXXaZr165Fl4HDRUsIgGSEEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACRjAFM6rGHDhkVbtnDhwqLL3HLLLUWXGTlyZNFloE23hF5++eX8BNqwYUOL66xevTpqampi3bp1H2T/AOjAigqhVatWxTnnnBMTJ06MSy+9NJ544on3rLN79+646KKLYsKECXH33XfHmDFjYtq0aVEoFEq53wCUWwi99dZbcdNNN8VLL73U4jpZ8CxbtixeeOGFWLp0aSxfvjwefPDBuP/++0uxvwCUawidddZZeSuooqLigF+6NXXq1BgwYEA+X11dHeeee26rvowLgI6tpL3j9uzZE2vXro0RI0Y0W57NZ8+IWtLQ0BD19fXNJgA6vpKG0Pbt22Pv3r3Rp0+fZsv79esXW7dubbHcrFmzoqqqqmkaOHBgKXcLgHIIoW7duuU/d+zY8Z5w6t69e4vlZs6cGXV1dU1TbW1tKXcLgHJ4T6hHjx7Rv3//93TdzuYHDx58wPBqDDAAykfJR0zIOiEsWrQo9u3bl8/v2rUrf18oWw4ArW4Jbdq0qdm7QStWrMhvsx1//PFxyimn5MtuvfXWGD16dEyZMiUmTZoUCxYsiC5dusT06dOL2RQAZaColtDGjRvzUMmmLGRee+21/L+zMGo0aNCg/KXWoUOHxmOPPRbjxo3LP886JwBAq1tCw4cPP6jxrrLebVmPNwA4EAOY0mFlQ0YV65JLLim6TPYM9HDJRiAplgFMact8lQMAyQghAJIRQgAkI4QASEYIAZCMEAIgGSEEQDJCCIBkhBAAyQghAJIRQgAkI4QASMYApnRYnTt3LrpM9t1Xbdn3vve9osvccMMNh2RfoBS0hABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACQjhABIRggBkEyXdJuGtufkk08uusxDDz10SPYFyoGWEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACQjhABIpqJQKBSijamvr4+qqqqoq6uLysrK1LtDGcnqXLH69u0bh8txxx1XdJkNGzYckn2BUlzDWzWK9ssvvxzPP/98jBkzJj760Y82+2zdunWxZs2aZsu6du0akydPbs2mAOjAigqhVatWxcyZM+OVV17Jp/nz58eVV17ZbJ2FCxfGnDlzYsKECU3LevbsKYQA+GAh9NZbb8VNN90U55xzTnTq1PLjpCFDhuRhBAAlC6GzzjrroNbbuXNnLF68OLp37x7Dhw8/rPfMASjz3nHr16+P2bNnx/Tp02PAgAFx1113HXD9hoaG/EHW/hMAHV/JQ2j8+PF5b5ylS5fGypUrY968eTFjxox8viWzZs3Ke1I0TgMHDiz1bgFQDiE0duzY6NOnT9P85Zdfnt+Se+SRR1osk3V2yLryNU61tbWl3i0A2qBWddEuVta62bRpU4ufd+vWLZ8AKC8lbwlt3Lix2XzWqsm6do8aNarUmwKgnFpCWWvmiSeeaJpfsWJF3gPu+OOPj1NOOSVfduGFF+YvsY4cOTI2b94cc+fOjerq6rjuuutKv/cAlE9LKGvlLFiwIJ+mTJkSr732Wv7fWRg1evzxx/NnQMuWLctbQXfccUcsX748evXqdSj2H4B2zNhxsB9jx0E7GDsOOqrWdJC54IILWrWtmpqaosts27at6DL//Oc/iy5z4oknFl0GWsNXOQCQjBACIBkhBEAyQgiAZIQQAMkIIQCSEUIAJCOEAEhGCAGQjBACIBkhBEAyQgiAZAxgCvvJvh+rWDfeeGOrjuGf//znVo1OXKwlS5YUXcYAphwuWkIAJCOEAEhGCAGQjBACIBkhBEAyQgiAZIQQAMkIIQCSEUIAJCOEAEhGCAGQjBACIBkDmMIHdOaZZ7aqXN++fYsu88YbbxRdZufOnUWXKRQKRZepqKgougxoCQGQjBACIBkhBEAyQgiAZIQQAMkIIQCSEUIAJCOEAEhGCAGQjBACIBkhBEAyQgiAZIQQAMlUFFozXO4hVl9fH1VVVVFXVxeVlZWpdwcOiQEDBhyWUbRbY8uWLUWXyc5ZKPYariUEQDJCCIBkhBAAyQghAJIRQgAkI4QASEYIAZCMEAIgGSEEQDJCCIBkhBAAyQghAJIRQgAk06XYAu+8804899xz8e6778bw4cPjqKOOet/1Vq9eHa+++moMGTIkhg4dWop9BaCcW0K33357nHDCCXHLLbfEbbfdFh/72MfizjvvbLbO7t2746KLLooJEybE3XffHWPGjIlp06ZFG/zGCADaU0vomGOOiXXr1kXv3r3z+YcffjguvvjiGD9+fIwePTpflgXPsmXL4oUXXsi/L+XFF1+MUaNG5etcddVVh+ZfAUDHbwldf/31TQGUyVo8Xbt2zW/PNZo/f35MnTq16Qu7qqur49xzz82XA0DJOiY89dRT+e23LGgye/bsibVr18aIESOarZfNZ8+IWtLQ0JB/E9/+EwAdX6tD6O23346rr746zjvvvDj99NPzZdu3b4+9e/dGnz59mq3br1+/2Lp1a4u/a9asWflXwTZOAwcObO1uAdDRQyhrqWS32I488sh48MEHm5Z369Yt/7ljx45m62fh1L179xZ/38yZM/PvIm+camtrW7NbAHT0Ltrbtm2LiRMn5rfelixZEpWVlU2f9ejRI/r37x8bNmxoViabHzx4cIu/MwuvxgADoHwU1RLKWjRZAGXPgR599NG8JfTfshbSokWLYt++ffn8rl27oqamJl8OAK1uCV1wwQV5B4M5c+bE0qVLm5YPGzYsnzK33npr3l17ypQpMWnSpFiwYEF06dIlpk+fXsymACgDRYXQRz7ykTjnnHPiT3/6U7Pll112WVMIDRo0KFatWhU//vGP47HHHotx48blQZR1TgCA/VUU2uBQBlnHh6yXXNZJYf9nTtCRNL5LV4w33ngjDoctW7YUXSY7Z6HYa3jRHROA0jjiiCPa7KF84IEHii5zww03HJJ9oWMzijYAyQghAJIRQgAkI4QASEYIAZCMEAIgGSEEQDJCCIBkhBAAyQghAJIRQgAkI4QASMYAppDI008/XXSZM844o+gy//rXv4ous2zZsqLLGMCU1tASAiAZIQRAMkIIgGSEEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJVBQKhUK0MfX19VFVVRV1dXVRWVmZenegzXj99deLLjNu3Liiy2zZsqXoMmPHjo3WqKmpaVU52q5iruFaQgAkI4QASEYIAZCMEAIgGSEEQDJCCIBkhBAAyQghAJIRQgAkI4QASEYIAZCMEAIgGSEEQDJd0m0aKNaxxx5bdJnHH3+86DL3339/0WX27t1bdBnQEgIgGSEEQDJCCIBkhBAAyQghAJIRQgAkI4QASEYIAZCMEAIgGSEEQDJCCIBkhBAAyVQUCoVCtDH19fVRVVUVdXV1UVlZmXp3ADhE1/CiR9F+55134rnnnot33303hg8fHkcddVSzz9etWxdr1qxptqxr164xefLkYjcFQAdXVAjdfvvtcd9998Xxxx8fnTt3jmeffTZuu+22uPnmm5vWWbhwYcyZMycmTJjQtKxnz55CCIAPFkLHHHNM3tLp3bt3Pv/www/HxRdfHOPHj4/Ro0c3rTdkyJA8jACgZB0Trr/++qYAylx00UX5rbbs9tz+du7cGYsXL86/TGvLli3FbAKAMvKBesc99dRTsXv37qiurm62fP369TF79uyYPn16DBgwIO66664D/p6Ghob8Qdb+EwAdX6tD6O23346rr746zjvvvDj99NOblme35jZs2BBLly6NlStXxrx582LGjBn5fEtmzZqV96RonAYOHNja3QKgo3fRzloqZ599duzbty+WLFnyP7vgjRgxIu+ocM8997TYEsqm/X9/FkS6aAO0P4e0i/a2bdti4sSJsWfPnoMKoEy2M5s2bWrx827duuUTAOWlqNtx27dvzwMoew706KOPxpFHHvmedTZu3Nhsvra2NlatWhWjRo364HsLQPnejstuqWXvBmXvAe0fQMOGDcunzKmnnhpjxoyJkSNHxubNm2Pu3LnRv3//+Mtf/hK9evU6qO0YMQGg/SrmGl5UCH3+85/PW0H/7bLLLsunTPZs55e//GU888wz8aEPfSgPpCuuuCJ/ufVQ/AMAKJMQOlyEEED7Vcw13CjaACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACQjhABIRggBkIwQAiAZIQRAMkIIgGSEEADJCCEAkhFCACTTJdqgQqGQ/6yvr0+9KwAUqfHa3Xgtb3chtG3btvznwIEDU+8KAB/gWl5VVXXAdSoKBxNVh9m+ffvi9ddfj969e0dFRUWzdM2Cqba2NiorK6NcOQ6Og/rgvGjL14csVrIAOvbYY6NTp07tryWU7fSAAQNa/Dw7sOUcQo0cB8dBfXBetNXrw/9qATXSMQGAZIQQAMm0qxDq1q1bfOc738l/ljPHwXFQH5wXHeX60CY7JgBQHtpVSwiAjkUIAZCMEAIgmTb5ntD72bt3byxfvjw2b94cJ510UgwaNCjKTfbvf/XVV5stO/roo+PMM8+Mjq6uri6WLFkSH/7wh2Ps2LHvu85rr70Wzz77bP5+wumnnx5HHHFElNNxyF4O/P3vf/+eMhMmTMjX7yiya8Hq1avjP//5T3z84x+PYcOGlWV92Ps/jkN7qQ/tIoQ2bdoUZ511Vn4CfuITn4inn346vv3tb8fMmTOjnNxzzz2xYsWKGD16dNOyoUOHdugQyk6kGTNmxO9+97t8JI1TTz31fUPohz/8YXz961/PP9+wYUN07tw5Hn300fjoRz8a5XIcsovuFVdcEZMnT44ePXo0Lf/kJz/Zpi46H8TixYvjpptuynt+ZS+0Z3+YnXzyybFo0aLo1atX2dSHxQdxHNpNfSi0A1/4whcKI0eOLOzYsSOfr6mpyXr0FVasWFEoJ1OnTi1MmzatUE42btxYuO+++wrbt28vTJkypXDhhRe+Z521a9cWOnfuXPjVr36Vzzc0NBROO+20wvnnn18ot+OQnRe1tbWFjio791966aWm+TfffLNw3HHHFW6++eayqg81B3kc2kN9aPMtoV27dsXChQvjBz/4QVOan3/++XHCCSfEAw88EKNGjYpy8uabb8bDDz8cRx11VHzqU5/Kx9fryI455pi49tprD7jOggUL8r/spk6dms937do1vvKVr8SVV14ZW7Zsib59+0Z7dzDHoVF2pyA7V0488cT8POlIsnN/f9l58NnPfjaef/75sqoP5x/EcWgv9aHNd0x46aWXoqGhIW9C7m/48OHx97//PcrNM888Ez/5yU/iuuuuy5+L/eY3v4lyl9WD6urqZoPdZvUju231j3/8I8pJNu7i97///fx21CmnnBIXXHBBh/5KlOyP1GXLljW7PpRjfdj1PsehvdSHNh9C2XOgzH//9dKvX7/YunVrlJNrrrkmv7+dPWx88cUX48Ybb4wvfelLsX79+ihnWR15v/qRKac60qdPn/xBfPaHSvbMYO3atfHcc8/FzTffHB1Vdg7s2LEjvva1r5V1fbjxfY5De6kPbT6EGoee2L59e7Pl2Xz37t2jnGQdELJbC42+9a1v5X/dZb2lyllWR96vfmTKqY5kt6BGjhzZNJ89sM5azDU1NdERffOb34yHHnoo76yRfWVAudaHb7ZwHNpLfWjzITR48OD8Z9YC2F/WVbnxs3LVpUuX/F5v9pyonGXdU9+vfmTKvY5kQ/l3xPqR9Y7NbjH98Y9/jE9/+tNlWx++fYDj0F7qQ5sPoawZnR3c/Z99ZBXsb3/7W5x33nlRLrLnYm+//XazZX/961/z2wv7d9kuR5MmTcqfA6xbt65p2a9//eu8+3pHu+gcyBtvvNFsPhsW8re//W2Hqx+33npr/rpCduE97bTTyrY+3Po/jkN7qQ9tvndc5v/+7//yW1FZ76ARI0bEvffem7989rnPfS7Kxc6dO+Mzn/lM3uc/O5leeeWVmDt3bv4ewNlnnx0dWfbuw+7du/OX8rIX9LLeT9ltyUsuuST/fOLEifkfJFmPoa9+9avx8ssvxy9+8Ys2d9vhUB+Hn/3sZ/nD6aw+ZLedsj/cspcZ//CHP0RHkfWS/e53v5v/f86+OTQ7BpkjjzwyrwflUh9+cBDHob3Uh3YzivaaNWti3rx58dZbb+W9PLJ7m+1lqPJSyR64ZidT9lde1iXzjDPO6PABlJk2bVq88847zZb17NkzP8kavfvuu3n9yF7ay96Q/+IXv5jXk3I7Dk888UR+sc16QGVdcr/85S93iC7JjX70ox/Fk08++Z7l2ddZ33nnnWVTH350kMehPdSHdhNCAHQ8bf6ZEAAdlxACIBkhBEAyQgiAZIQQAMkIIQCSEUIAJCOEAEhGCAGQjBACIBkhBEAyQgiASOX/AS7EY3NPwgLQAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visual verification\n",
    "image = np.asarray(values[1:], dtype=int)\n",
    "plt.imshow(image.reshape(28,28), cmap='Grays')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dabacd4",
   "metadata": {},
   "source": [
    "## 6.1 Hyperparameter Tuning: Learning Rate Impact\n",
    "To optimize the network's performance, we evaluate the impact of the Learning Rate ($\\eta$) on the final classification accuracy. The network is trained across a sweep of different learning rates `[0.01, 0.1, 0.2, 0.3, 0.6, 0.9]` while keeping the hidden nodes constant (100 nodes). \n",
    "\n",
    "The results are plotted to identify the optimal step size for the Gradient Descent algorithm, avoiding both slow convergence (values too close to 0) and divergent oscillations (values too close to 1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "b3be2fde",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting Learning Rate sweep. This may take a few minutes...\n",
      "Training network with Learning Rate: 0.01...\n",
      "Performance for LR 0.01: 0.8683\n",
      "\n",
      "Training network with Learning Rate: 0.1...\n",
      "Performance for LR 0.1: 0.9249\n",
      "\n",
      "Training network with Learning Rate: 0.2...\n",
      "Performance for LR 0.2: 0.9274\n",
      "\n",
      "Training network with Learning Rate: 0.3...\n",
      "Performance for LR 0.3: 0.9178\n",
      "\n",
      "Training network with Learning Rate: 0.6...\n",
      "Performance for LR 0.6: 0.8587\n",
      "\n",
      "Training network with Learning Rate: 0.9...\n",
      "Performance for LR 0.9: 0.8326\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAscAAAHVCAYAAAAdAFYjAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlcelbwAAAAlwSFlzAAAPYQAAD2EBqD+naQAAbd9JREFUeJzt3Qd0FGUXxvELBEIJvffeQXqT3lEB6UWpCkhRUIqAgoCIIApYsCAgvQkIUqSDVOm9996khhoI7Hfu6+5+m5BAEpJs+//O2UN2sjOZmd2QZ9+9c98YFovFIgAAAAAkJucAAAAA+A/hGAAAALAiHAMAAABWhGMAAADAinAMAAAAWBGOAQAAACvCMQAAAGBFOAYAAACsCMcAotSECROkevXqki9fPmnVqhVnG9Hm9ddfl27dunHGAYQL4RjwMn///bfkyZPHfsufP7+UL19eevfuLZcuXYrUnzVv3jx55513pFmzZjJ37lz54osvInX7iHqHDx82r5NffvnF7U73yZMn5cKFC077+fq6d/xdK1y4sNSvX19mzpwZ4W3u3LnTbEt/jwFEDZ8o2i4AF3X37l05cuSIfPnll+YP9ePHj80f3D59+sjkyZNlx44dki5dukj5WQsXLpSMGTPKu+++GynbQ/R7+PCheb1cu3bN7U7/kiVLxNfX12k///Tp03Lr1i17kL1+/bpMmTJFmjdvbn7Pvv7663Bv8/79++b50N9jAFGDcAx4qbRp05oRKFWwYEFJlCiRNGjQQH744QcZOnRopPyMy5cvm+0CzpA1a1ann3gfHx/775kqW7as7N69W7799lvp16+fJE6c2Kn7B+BZlFUAMEqVKmX+PXr0qP2M6IjXW2+9JUWKFJGSJUtKjx495MqVK/bvP3nyxPzh/+6778zH7y1atJBChQrJiBEjzPJ169bJsWPH7B8r//XXX/Z1p0+fLrVr15ZXXnlFKlasaAL5vXv3XrjtadOmyapVq8z39uzZY8o1qlatavZPyzaePn1q1p8zZ45Uq1bN7HuvXr3kwYMHQZ7pU6dOPfORd6NGjWTRokVBHuf4sxYvXiw1atQwj+3QoYMJ/8EFBgbKuHHjzLHp4/TfSZMmicViCfKYX3/9VV577TVz/FWqVDFvSnR5aA4ePGj2Y+rUqc98T0d1tTxm1KhR5r7+LP2Z+rP1+PXnDB8+3Iw6RpV///1XPv30UylXrpx5nvRTiRUrVjxTf24733nz5jWvuY4dO5rn11H//v2lTJky5lMN/YTj1Vdflbp165rvaQlQ3759TQlQ+/btzfHpc6LPTVhqjsOzvm2f9fkpWrSodO7c2ZxrLRXS10pE6bb0udayDxv9vXJ8PdrOob6+HT+Jefvtt83XnTp1sj9W99FGX/96/4033jDb0N8t/X189OhRhPcX8DoWAF5l4cKFmtIsEyZMCLJ869atZnn79u3N/W+++cYSK1Ysy4cffmhZv369ZcWKFZaKFStaMmbMaLl06ZJ5zOPHj806LVu2tFSpUsVse/HixZZp06ZZDh06ZClXrpwlZ86c5mu9+fv7m/W6dOlitj1o0CDL5s2bLZMmTbKkTJnSUrRoUcu9e/eeu+1Zs2ZZ5s2bZ77XtWtXs61NmzaZbSRIkMDSr18/y9ixYy0dO3a0bNy40TJlyhSLn5+fpXPnzkGONyAgwL5fetPH9u7d2xIjRgzL9OnT7Y+z/ayePXuac6Pn4s8//7RkyJDBUqJEiWe2Wa1aNUvChAnN+dNj031u27at5eeffzaPefTokaVGjRqWZMmSWX788UfLtm3bLFOnTrWkTZvWUr9+fcvTp09Dfe7y5s1rKVWq1DPLR44cafbxwIED5v7gwYMt8ePHt/zyyy+WnTt3WlauXGnp27evpV27duF8tVgsu3btMtvWbYbm2LFjZv/1+dPzpa8lfW5jx45tngub69ev28+37uvSpUsttWrVsiROnNhy9uxZ++PeffddS/LkyS3vvPOOZcSIEeb5HTBggPle6tSpLQ0aNLC89tprlrlz51r++ecfS4sWLSwxY8Y0P9dR7ty5LQ0bNgyyLDzrf/rpp2b5Z599Zp5LfV3XrFnTUqFCBUuxYsVeeO70uUqfPv0zy998801zTs+dO2dfpq/34K9H/bn6e/Lrr7+ax9y+fdvsg66rryfbY/W8qidPnljq1atnzueoUaPM8ejvi/7O6n7r9wG8GOEY8DIhhWP946oBVIPh33//bTl8+LAJBRoIHd29e9eSJk0ay3vvvRckwGoYvHHjhv1xtoCnf5Dz588fYgjv379/kOUaOh1D2PO2bQusGnIcffTRR+bxb7/9dpDlvXr1svj6+lru3LnzwvOjQalAgQL2+7afpaHD0YwZM8xy3W+b4cOHm2XLly9/Zrsaih2DrJ5nR2vXrjXL9fkJjQZuxxBso/tbunRp+/1ChQpZGjduHOo+RHY41tdOunTpTHgLft71+Qi+3JE+zylSpLD06dMnSDjW1+L333//zGtKw60+l6dPnw7ypkTfXAV/3kMLx2FZX7+vvwPdu3cPsr4Gdd23iIbjRYsWmcCrvxthoW/ysmTJ8szvSUivEw3R+j39GY62b99u9llfswBejLIKwEvpR8v6kWz27NklderUcvbsWXMVvX4MqyUJ+vFs8AvpEiRIYD6C1gudHNWqVUuSJk1qvx8jRoxQf66tbKFNmzZBluvH8Tly5HimrOF5227SpMkzH1ffuXPHlBEEXx4QEGAukHK0adMm8xG5lmTox/x6PvTYtLTEsQxCNW3aNMh9XUfpxVE2s2fPlpw5c5rWdcHFjh3b/Pv777+bc67n2VGFChUkWbJkz5xbR9oKT7czfvx4+7KtW7fK/v37gzxXGTJkkNWrV5tuIXpBXfB9iExaZrBmzRpp3LjxM/XlWhagz8fGjRvNfX1NablHvXr1TMmJnvMCBQqYxwQvrdDzbyshCP68ly5dWjJnzmy/HydOHLM9x+fiecKy/tKlS83+almRIy33yJQpk4SVY7lEypQpTamLdrHQ10pw27dvN6UeWm6irQ91HS2rOHPmzDNlQSHR15b+LmtJhaNixYqZC2Of99oC8H9ckAd4qa5du5rwEjNmTBPKUqRIYf+e/jFWDRs2NKHE+imT/Y998CvlwxMWLl68GOo6Glgca55ftG0NgY6SJEny3OXaLcBmwYIF5vhbtmwpQ4YMMRco6sVT33zzjQmfWvOs9200XDjSc6YcuzicP3/eXNz4PHpu/f39TShUtnOrNz2vIdUx22i40tpb7XgwbNgwE3Z/++0386bFMbxr/XK7du3M86fdGjRsaSjTOlV9bGTSN1W677NmzZKVK1faXyf6r74hUbZj0prdiRMnyueffy4ff/yxeV709af7Fjz86X7aznFwwZ8LpY/V+vawCMv6tn0O/lqyLXN80/E8yZMnl/nz55vzoW3ltIZa3wBqLXSJEiXsj9Nzp2/q9DnT86MdY/T5/emnn8zzqecyXrx4EX5tXb169bmvLQD/RzgGvJRjt4rgEiZMaP7VIGP72lHwkeG4ceOG+efatnfz5k0T9hxpeA3+85637VixYoVrueNosHYLyJ07tzlGR6GN0IVlmzpy+qKWZ3p8GnxCurDO9v3n0dCro4l6cZaOqs+YMcMEY8f1tEuDXkiogUhHbZcvX24uctNPBrZt2/bckf3wsv1cHeXVfQvttaahTS9U7N69uwnGwS/m008NXvZ5Dz7a/zLr+/n52V+nOhrrSJe9KKiG1K1CR8q1W4WGV/3UQy+ytG3n+++/l/Tp05sLVfUNg43tDUZYn4ssWbKYT35CEtlvjABPRTgG8Ay9On/kyJGmDEE/Lo9M2nlAuypoeNOPlx1HlA8dOhRts+jpR/ka2hxptwwNki9z3nTUWUfwHD+2D/4YHfnVkXrH0fqw0rIWHfnUn6MjzRo6Q+sjnSpVKjM6rjcNeIMGDTIjvaHtW0RoqNX90fZkob3Zsn3ioKPxwc+5ln/oMbgaLZ9Qa9euDXJc2uXi+PHjL/yEIDQahrW/sY4Q6xs0LW+yvR71OXIMxtphwrHDi+ObhpA6m+hrS7u76Ju0yOpVDngjao4BhNgCS0clP/jggyBhUUex/vjjDxOcI+rNN9807ct0Rj6tlVU6UYLW/uooW8+ePaPlGdGaXx1V3bx5s7mvQVP3IVeuXBHepgYdDSYa+m1tujQQagmHfrSutN2ZjkpqK7ATJ04EGTXXkg590/A8Gp7atm0ry5YtM4/X4KZvOBxp+YTWr9ra2mno1xFj/YhfA7PStm66rj72ZegotIY8DbnaMs+x5Ebf7Oj2dTReg59tpN42uq51xlpC4Ar9iIPTc1qpUiWzf9rGzxZgtZ1h8FHu8NJ+4loHrO31dBTa9nrUyXi0ftv2/Gi7wOBlRToyrK8BfTMSnJ7/NGnSmODtWMOtP0NDc2jt6gAERTgGEGLg0TCnI5L6cbl+XKt/pDVc6UfzlStXjvBZ0zpKDdxab6kXyukIl5ZXaDjUYPgy4TQ8Bg4cKHXq1DEfc2sNqQYevZDuZY5NR1D1Ij+tpdXj0FFSrWXVvrPaT1fpz9JArqPG2ptYQ6MGVr0AS0Oj9qZ9EQ3xWgKwb9++EEeNNdS9//77Jqhr8NTzqwFZRyFtH+NrcNYL0M6dOxemY9Nw5diH13bT5Rr29GIv7YutF0/qa0VfM1o6oBe62Wap07pk/bl6DrSEQN8gaLB21YlitHylePHi9tepPjd6vvV8vuzFjdqTW98UakBWOkOllsdob249PxqCNUDra9SRvm4+++wz0xdcz7Njn2N9Hf3zzz9mXd1n3U8Ny/pa1E8zdHsAXiyGtqwIw+MAeAgNSRqINLiFZXYu/S9CSx70Xw00wetVdYQqtBIBvUBNJ3IIbWRQR8f0IiENkyFdfBXatnV0UretIcCxNtV2bFo24FgTaluuYSJ+/PjPnA8N5hp+dORaA6rebB+lh/azNOTpxYMaSELad92u1vzqdkObwlhH4vVjeg2HoV18FhodddZzG/xYHelFY1rOoAE8eA2vPp8ajnUUO6SLzhz3USdMCU3w5+f27dvmfOrrK7T90vPiWGKh5Tt67m37oa8JPe8hjdBqSYPWzgYvz9DXqI5QaycQG91vPfeOJQbhWd9x5FVHjfX1r/XK2pFEXx9a9/08Gki1/CGkbSo9/9opw/H3Q/fB9rrRAH7jxg1zXwOuY8mFPk73WV8DIb0Gdbl+X481IuU7gDcjHAMAEEZaLqNBVeu3tUQGgOchHAMAEAKt69aRX62/1xFjbcXWvHlzU86i9dRasgDA87hEzbF+3KkXj4T1imX9OFPfvevFC6G1XdKPDPUxO3bsMB9JAQAQHloT/uuvv5o6ai2p0JpyLTPRnsQEY8BzOXXkWOvh9GIfvZBD6+a0Puurr74yV8iHRmeD0qb9Ws+n/2FpLeAvv/wSpAH+3r17zX2tE9P/0LRuUd/5a1/RsPamBADAVr+ro8Zao+84WyMAz+TUkWOt2dKwqxeW6EdU2vxcZ+3asmVLqP9Bab9OnWZWg7R+tKXraF9Ux5mNtHWQhm0NzjpyrNvWq+B//PHHaDw6AIAn0Avj9IJMgjHgHZwajrX9jM6oZLtquF69embmIFtbmuB0BFivvu3YsaN9hiOdblOvcHZcR2db0jZRtqlf9Qp1fYwuBwAAAFxuhjwNudpiKHjfxZIlS8quXbtCXMfWqkY/3rLNSa+lGdo2SEeIbbRpu04koO/0dQRZe6pqy6jOnTuHuj9aR+Y4TafWNWsLHe3rGplTrQIAACByaHWwtlrU9oeO7Q7dMhxr8FQaPh3pfdv3grM1jdfSC714T3s3/vTTTya8Oq6jU2hqyNbZqnTEWC/M0wbrwWcacqQN1bXMAwAAAO5Fmzs8r2e7W4Rj2+xCOvLrSLtPaFP00EybNs1cgDdv3jzzWJ2BSWdi0tFk2zsILbWw1RzrtvSEaVjWljz9+vULcbsapLt37x6kkb2GaW0i76qzNwEAAHgzf39/M5GOZsHI4rRwrC1xdPjbFmpt9P7zRng17OrIsd5shg0bZqbctJVraIs3nZrTFrL1Z9WtW1cWLFgQajjWWZRCmsVKSzkIxwAAAK7Hdn1ZZJbAOu2CPJ3C9dVXXzWB1Ubbs2n/yOrVqweZ6tOxBjl4X2PtQqGdKrRjhS3MaujWUWNHOnqs88wDAAAALjdyrHR0V4OwljSUKVNGfvjhBzNHfIcOHYKMCm/evFn2799v7n/55ZemPKJy5cpmXvqBAweaeuLSpUub72sf4/bt28snn3wiT548kWzZspkL8pYuXSqLFy922rECAADA9Tk1HFesWFHWrFlj+g9rv+OCBQvKlClTxM/Pz/6YnDlzyqNHj+z3BwwYIN9//72MGjVKkiRJYlq4acmEI91eqVKlzNSfc+fONfXHmzZtsgdoAAAAwOVmyHP1Am+dDUkvzKPmGAAAwDvymlMnAQEAAABcCeEYAAAAsCIcAwAAAFaEYwAAAMCKcAwAAABYEY4BAAAAK8IxAAAAYEU4BgAAAKwIxwAAAADhGAAAAAiKkWMAAADAinAMAAAAWBGOAQAAACvCMQAAAGBFOAYAAACsCMcAAACAFeEYAAAAsCIcAwAAAFaEYwAAAMCKcAwAAABYEY4BAAAAK8IxAAAAYEU4BgAAAKwIxwAAAIAV4RgAAACwIhwDAAAAVoRjAAAAwIpwDAAAAFgRjgEAAAArwjEAAABgRTgGAAAArHzEye7fvy+LFy+WK1euSMGCBaVixYovXOf8+fOybt06uXXrlpQuXVqKFi0a4uOOHTsma9eulfjx48sbb7whiRMnjoIjAAAAgKdw6sjxpUuXpFChQjJkyBDZsWOHNG7cWFq0aPHcdaZMmSK5c+eW2bNny65du6RmzZry8ccfP/O4vn37SpEiRWTNmjXmVr58eTl58mQUHg0AAADcXQyLxWJx1g9v1aqV7N+/X/755x/x9fU1X2tYnjt3rtSrV++Zx9+7d09SpkwpgwYNkl69epllJ06ckLx588rq1aulXLlyZtnkyZOlXbt2smnTJilevLg9iAcGBkrGjBnDtG/+/v5mpPn27duSKFGiSD1uAAAAvLyoyGtOGzl+8uSJ/PHHH9K6dWsTjFWBAgVMwP39999DLZN48OCBVK1a1b4se/bskiVLFpk5c6Z92YgRI6Rp06b2YKzSpk0b5mAMAAAA7+S0muNz586ZkWAtkXCUJ08e2bp1a4jrZM6cWWLFiiXbtm2z1xlfvnzZbOvAgQPmvm5z37590q1bNzMivXPnTkmXLp1Ur15d/Pz8Qt2fgIAAc3N8J6J0tFlvAAAAcC1RkdGcFo7v3Llj/k2SJEmQ5Xrf9r3gkiZNKgMGDJAPP/xQ9uzZIylSpJA5c+ZIpkyZ7OvcvHlTtFJESys04OoFexMmTJD3339fVqxYIfny5Qtx20OHDjXlGsFpXXOCBAki4YgBAAAQmXRQ1GPCsXaQcByhtdGakeeF0f79+5uL8PQiOy2xmD59ugm1tnBs265+b/v27RIzZkwTlrULhtYpa2eMkOgFfN27d7ff1/3SMgy9qI+aYwAAANcTPEe6dTjWEgmtNdYL6hzp/Zw5cz533ZIlS5qbevr0qWzevFnatGlj7idLlsxctKdhWIOxihEjhlSqVMl0ugiN7out9tmRj4+PuQEAAMC1REVGi+nMg9Hew9OmTTMBV50+fdr0Ja5fv779ccuXL5eJEyfa7x8/fjzIdsaMGWP6Hbdv396+rEGDBqYu2ZHWMefKlSsKjwgAAADuzqmt3HSU+NVXXzWTf+hIsHac0FHjv/76y1x4p7Qlm44Ma5s39f3338v8+fPNSPCRI0fM17/99pvpTmFz9epVs13tZFG2bFnZsmWLuThPSzG0VVxY0MoNAADAtUVFXnNqOFb//vuvzJgxwz5Dnk4EYgvGSif7OHv2rPTo0cO+bOPGjbJy5Upz8Z6OMusFecHdvXvXbPfMmTPm+w0bNpTkyZOHeb8IxwAAAK7NI8OxqyIcAwAAuDaPmgQEAAAAcDWEYwAAAMCKcAwAAABYEY4BAAAAK8IxAAAAYEU4BgAAAKwIxwAAAIAV4RgAAACwIhwDAAAAVoRjAAAAwIpwDAAAAFgRjgEAAAArwjEAAABgRTgGAAAArAjHAAAAgBXhGAAAALAiHAMAAABWhGMAAADAinAMAAAAWBGOAQAAACvCMQAAAGBFOAYAAACsCMcAAACAFeEYAAAAsCIcAwAAAFaEYwAAAMCKcAwAAABYEY4BAAAAK8IxAAAAYEU4BgAAAKwIxwAAAICVjzjZ4cOHZezYsXLlyhUpWLCgdOnSRfz8/J67zuLFi2XZsmVy69YtKV26tLRr107ixIkT4mOXLl0q48aNk9q1a0ubNm2i6CgAAADgCZw6crxz504pVqyY3LhxQ8qXLy9z5swx/wYEBIS6To8ePaRly5aSPn16qVixokyePFlef/11efr06TOPvXDhgnTo0EHWr18vu3fvjuKjAQAAgLuLYbFYLM764TVr1hQfHx8zEqyuX78uGTNmlJEjR0rHjh2fefy1a9ckZcqUMm3aNHnrrbfMstu3b0uGDBnkt99+k8aNG9sfq2G5SpUq0rRpUxkzZoxUqlRJvv322zDvm7+/vyROnNhsP1GiRJFyvAAAAIg8UZHXnDZyrKPDq1evlkaNGtmXJU+eXKpWrSp//fVXiOvoSLDKnTu3fZmekNSpU8uiRYuCPPbzzz+XhAkTSqdOnaLsGAAAAOBZnFZzfPbsWQkMDJRMmTIFWa73165dG+I6uXLlMu8Kfv/9d1OOobZv3y6nT5+WNGnS2B+3bt06U8e8a9euMO+PhnXHcg59J6J0H/UGAAAA1xIVGc1p4dgWROPHjx9kuV6M9/DhwxDXiRcvnkyaNEnatm0rK1askBQpUsiJEyfMRXm27WlpRosWLUw4TpUqVZj3Z+jQoTJo0KBnlmvATpAgQTiPDgAAAFHt3r17nhOOtRxC3bx5M8hyDbdJkiQJdb169epJ5cqVzQV2Dx48kLJly5paY9s6s2bNMqO+WoOsN6Ujy1rXfP78eTPqHDPms9Ukffv2le7du9vv6za0/rlIkSLUHAMAALgg2yf9HhGO9SK6pEmTyt69e023CRu9/8orr7wwWGunCts7ho0bN8qAAQPsF/kFHzHesWOHqVNu1qyZxIgRI8Rt+vr6mltwesGg3gAAAOBaoiKjOe2CPA2pb7/9towfP970K1Zaa7xt2zZTFmHzyy+/BBnRXblypX0IXTtS9OrVy5RivPvuu2ZZ9uzZzUV+jjcN0zly5DBfhxaOAQAAAKcOiQ4ZMsT0Os6bN6+5bdmyRT755BPTgs1GL7jbvHmz/b6WUhQqVMhcnHfs2DH7RB86Cg0AAAC4bZ9jpT9eR4t1hrwCBQpI1qxZnymJ0ElCqlevbl+mdcpbt241dcbFixeXWLFiPfdn6Giz9kfWUB1W9DkGAABwbVGR15wejl0V4RgAAMC1edQkIAAAAICrIRwDAAAAVoRjAAAAwIpwDAAAAFgRjgEAAAArwjEAAABgRTgGAAAArAjHAAAAgBXhGAAAALAiHAMAAABWhGMAAADAinAMAAAAWBGOAQAAACsf2xeAtzh7+YZcu3U3zI9PkcRPMqVJFqX7BAAAXAPhGF4XjHM36C8PHwWGeZ24cXzkyB+DCcgAAHgByirgVXTEODzBWOnjwzPSDAAA3BfhGAAAALAiHAMAAABW1Bx7MS5MAwAACIpw7KU8/cK0J0+eyvXbd+XqjTty9eYd8++VG/6y9+h5Z+8aAABwYYRjL/UyF6Y5IxxbLBa5ez/AGnT97WHXMfw6/qv7qesAAACEB+EYTvPo8X9h2xZor1z3/3/4DSHwPgx4HK7tx4gRQ5InTiCpkiWUVEkTmn/V7yt2hHtfx85bL92aV5U8WdKEe10AAOA+CMeINDpSe+vO/ReM6v5/+U3/++H+GQni+f4/7FoD7//DbyJJbe4nMvc1GPv4xAqy/s7DZyMUjn+Zu87cCmRPJ42rFZNGVYtKvmzpwr0dAADg2gjHCJcFa3fLis0HQxzZ1RHfwCdPw7W9WLFiSsokfvZA6zjKm1qXOdxPmTShCcfO8Oor2WXrgVOy/8RFcxswZqHky5ZWGlctZsJy/uwEZQAAPAHhGOEyaOziFz4msV+8IKE2VdL/Qq7jqK7t+0kTxZeYMV2/o+APHzeTrOmSy59r98icVTtl+eaDcvDkJRl0cpEMGrtI8mZNa0aTNSjr6LKWdAAAAPdDOEa4lHklm+TMmMoebu2ju9bQmzKpn/jGie2RZzVpogTSps6r5qblIwvW7pHZq3bI8s2H5NCpSzJ43GJzy505tb304pWcGQjKAAC4EcIxwmX0x82laJ5MbnvWUiTxMy3pwtvCTtdzlCRhfGlVu4y53b77QBau2yOzV+6Upf8ckCNnrsgX4/8yt5yZUtlLLwrlIigDAODqYljodxUif39/SZw4sdy+fVsSJUoknkYvTCvWYki419sx9VO3DsdRPfmJvwbl9Xtl9sodJigHOITwHBlT2UsviuTOyIgyAAAumNcIx9F4sl2JN4fj6KJBefGGfab0YsmmA0Fa0WVLn8JeelEsb2aCMgAALpLXKKsAokgiv3jSvFZJc7tz76EJynox318b98nJC9fkq0nLzC1r+hTSqMp/I8rF8xGUAQBwJsIxEA0SJogrzWqWMLe79x/KXxv3m9ILDcynLlyTr6csN7fMaZPbSy9K5s/CiDIAANHM6T20fv/9dylbtqzkyJFD6tevL4cOHXru4+/evSv9+/eXkiVLSq5cuaRVq1Zy/vz5II85evSodOrUSQoVKiTFihWTbt26ydWrV6P4SNzzwrTwCOnCNISfX/y40qR6cZn91Xvy78oRMntYB2lSvZjEjxtHzly6LiOmrpDSbYZJljqfSI9Rs2XzvpNMhQ0AQDRxas3xvHnzpEmTJvLDDz9ImTJlZMSIEbJ06VI5cOCApEyZMsR1XnvtNTl37pz8+OOPkiJFChk1apSsWbNG9u7dKwkSJJAnT57IK6+8Il27djXbvH//vvTs2dPUomzbtk3ixo0bpn3z9JpjdfriNanaaZT5iP+DJpWlTd1XI+3CNITf/YePZMnG/ab0Qi/qu/cgwP69jKmTSkNr6UXpglndojc0AABRzeMuyCtatKi5jRs3ztwPDAyUtGnTygcffCCfffbZM48/c+aMZMmSRZYtWyY1atQwyzQMp0mTRgYOHChdunQxy54+fRokPBw7dsyMMmuIrlSpUpj2zRvCsX6kX/vD0eYj/7OLhpr2ZHANDx4+Mt0utPRCg/Ld+/8PyulTJTGlF42qFpNXX8lGUAYAeC1/T7ogTw9m165d0rt37//vjI+PVK1aVdatWxfiOjoKrBImTGhfFitWLIkfP778/fff9nAcfFTt8eP/ugTEju2Zk1NE1PDJy8y/7zUoTzB2MfHixpH6lYuYmwZlnZFPu14sWLdXLly9Jd/NWG1u6VImkYZVipgR5bKFsj83KEdlCzsAADyF08LxhQsXzL866usoderUsmfPnhDXyZkzp2TLlk2GDRsm06dPN2UUOup89uxZSZcuXYjr6MD4J598YmqaS5QoEer+BAQEmJtjeLeNZuvN02zef0rW7TwmsX1iSZfGFT3yGD1FbJ+Y8ka5Auam7eBWbDkkc9fskoXr9srFf2/JD7PWmFvaFImlXqVCpvOFBuVYsWIGCcb5mwwK9+QnB34fQEAGALisqMgvTgvHWvpgdsAn6C7o6K6WSoREHzt//nx57733JHny5OLr6yulSpWSunXrypUrV0JcR4Px6tWrzchynDhxQt2foUOHyqBBg55ZrqPbGsI9Tf+xq82/NYpllctnT8jls87eI4RV2vgi77+RXzrUyCNbD1+U1btOyfp95+TStdvy85x15pY8UTypWCizVC2SRQrlSC3HL9wIVzBW+vj1/2yV3BlT8OQAAFzSvXv3PCcc2y64u3btWpDlej+0i/FUwYIFZdOmTfLw4UNzS5IkiZQvX14yZXp2YgqtQx49erQsWbLE1DY/T9++faV79+5BRo4zZswoRYoU8bia46Nnr8javf+l4SHdmkr+bCGPusP1vVpG5MO2IgGPHsuqbUdk7updsmDdHrnu/0D+WH/Y3FIlTSjlCmeP0Pbz5s3HpC8AAJdl+6TfI8JxqlSpJHPmzLJx40Z588037cs3bNggderUeeH62nVCb1qesXnzZhk7dmyQ7+sosHa/+Ouvv6RcuXIv3J6OQustpNHq4KPb7u67GWtMuUnt8gWlUC5mu/ME+hqtW7GwuT16HCirth42F/PNX7tbrt68I3+s2R3h7Xra6x8A4Dl8ouBvlFP7QekFdOPHjzc1xlpm8f3335v64Q4dOtgf8/HHH0u1atXs97XG2FaTfPnyZWnZsqUULlxY3n77bftjhgwZIt98840JxjqqjP+7fO22TFr8z3/ntlVNTo0HihPbR14rW0B+G9Bariz/Rpb+0FXerFjI2bsFAIBbcOqQUI8ePeTSpUtSunRp03VCyxdmzpwpefPmtT/mxo0bJgTbaI3xu+++K8ePHzdlFfXq1ZPZs2fbO1Ho4/v162c6WDRv3jzIz/v666+fWeZt9MKtgEeBplduucI5nL07iGJ6wWXNMvklZdKE8ufakC90BQAALhKOte3UyJEjTfcJ7U+nk3rEiBHjmUD76NGjIDXHW7dulVu3bpkAHPwiO61B1klCQpIsmXe3pbpz76H8NGetfdQ4+LkGAADwdi5RTKgBN7SL8JImTRricg3BoQXuDBkyROr+eYpx8zfIrTv3JVem1FK3Ah+zAwAABMcctF7iceATGTV9pfm6Z8vqQXrgAgAA4D8kJC8xa/k2OXflpqROnkhavl7a2bsDN7F4/V7T2QQAAG9BOPYCGm6GT15uvu7WrIrE9WUabYTNZ2MWSoNev8iV65HfRxIAAFdEOPYCy/45IPuOXxC/+L7SsWEFZ+8O3IhPrJgy/+/dUqDpIJm7aqezdwcAgChHOPYCtlHjDvXLS9JEnjcVNl4sRRI/iRsnfNff6uMXjuoir+TMINdu3ZVGvcfI2/3Gy43bkT9VJwAAriKGhYLCUKcjTJw4sWkx587TR287cFpKth5qRgBP/jlEMqbx7nZ23uzs5Rsm5IYnUGdKk8zMuPf52EUydOJSefrUImlTJJZx/VrK6+UKRun+AgDgjLzmEq3cEHW+nvLfqPFbtUoSjL2cBl29RWTGvS8615M6FQpJ6wET5MiZK/LGh6OlXb1yMuLDRpLIL16U7C8AAM5AWYUHO37uqsxd/V+daM+WNZy9O3BzpQpklV3T+slHb1U1E8ho3+xXmg+WNduPOHvXAACINIRjDzZy2krzMfjrZQtIwRzpnb078ADx4saRkd2byJpfukvW9CnkzKXrUqXjSOn2zSy5//D/M1kCAOCuCMce6uoNf5mwcJN9qmggMlUslkv2TO8v7zX4r/vJ9zNXS+G3Bss/e09wogEAbo1w7KFG//63PAx4LCXzZ5EKRXM6e3fggRImiCu/fPK2LP2hq6RPlUSOnb0q5dp9LX1Hz5OAR4+dvXsAAER/OL5w4YKsX7/+ZTaBKHD3/kMZ/fsa+6ix1ocCUaVmmfyyb+ZnZuZFLeMZNnGpFG/5pew6fJaTDgDwjnB848YNqVWrlmTIkEEqVPj/pBK1a9cmLLuA3xZskpv+9yVHxlRSr1JhZ+8OvID2z578eVv54+uOkjJpQtl/4qJpIagt4B4HPnH27gEAELXhuGfPnhInThw5c+ZMkOUfffSRfPHFFxHZJCKJBpERU1eYr3u2qC6xYlE5g+hTv3IROfD7AGlYpagEPnkqA8YslFff+UoOnrzI0wAA8NxJQNKkSSM7duyQ9OnTm4/sbZvQBsxp06aV+/fvi7tz10lApi/damYxS5UsoZxe8KXpLgBEN/0/YcaybdLlqxly68598Y3jI0M615MPm1flDRsAwKXzWoSGFXUHEiZMaL52rGfV5T4+zCvizEAyfPIy83XXplUIxnAa/X9BJ57RUeTXXi0gAY8Cpee3c6Rih29M/20AAFxVhMJxyZIlZfbs2UHC8dOnT01JRdmyZSN3DxFmK7Yckj1Hz0uCeL7SqVFFzhycLl3KJLL4u/fNdNN+8X1l454TUqj5YPl5zlr7J04AALiSCA3zfvXVV1KjRg1Zs2aN+QPXq1cvWb58uRw/fpwL8pzINmrcvl45SZY4gTN3BbDTN9Dv1isnVUvmkbaDJsnfO45K52HTZd6aXTK+fyumNQcAuP/IcenSpWXz5s3i6+srBQsWlKVLl0rhwoVl69atUrRo0cjfS7zQjkNnZNXWw6ae86O3q3HG4HKypEshq37+SL7r2VTi+cY2n3QUaDpIJi7cxCgyAMC9L8jzBu52QV6zvmNl1ort0uK1UjJl8DvO3h3guY6euSKtB06QzftOmft1KxSSMZ+8LWlSJObMAQDc74K8J0+eyO7du59Zrsv0e4heJ8//K7NX7TBf92pVg9MPl5crc2rZMO5jGfZ+fYkT20cWrNtjRpFnr/zvdQwAgLNEKBwPHTpU5s6d+8xyXab1yIheI6etNDOT1Xo1v7ySMwOnH25BS4B6t6kl26d8IoVzZZTrt+9Jkz6/mk9Brt+66+zdAwB4qQiVVejMeFpfnC5dumemk9ZuFadPnxZ35y5lFf/evCOZa/eVBwGPZfUv3aVy8dzO3iUg3B49DpQhv/0lQ35bIk+ePJU0yRPJr5+2kDoVCnE2AQCuX1Zx8+bNIP2NbXTZ1av0MI1OP/7+twnGxfNllkrFckXrzwYii5ZWDHqvrmye0FvyZk0rl6/7S93uP8k7gybJ7bsPONEAgGgToXBcqlQp+fbbb59ZPmrUKNMDGdHj3oMAGf37GvP1x61qhviGBXAnxfNlkZ1TP5WeLaub1/OEhZukYNNBsmrrIWfvGgDAS0SorOKff/6RKlWqSJEiRaRChQqmDdO6devMBXmrV6+WMmXKiLtzh7KK0bPWyAdfz5Rs6VPI0T8GMy0vPMqG3cel9YAJcvLCNXO/S+NK8lXXBmaSGwAAXKqsQsOv1hznzJlTFi9eLEuWLJFcuXKZZZ4QjN1BYOATGTFthfm6Z8saBGN4nHKFc8ieGf2lc+P/Znv8cfbfZna9jbuPO3vXAAAejD7HbjpyPHPZNmn+6ThJmTShnFn4pcSLG8fZuwREmRWbD8o7gyfL+Sv/Xe/Qs0V1+bxjXYnrG5uzDgBezN9VRo5ttKfxrVu3nrkhamkZi22q6A+aViYYw+NVL51P9s38TNrUKWNe/19PWS7FWg4xM0MCABCZIhSODx8+LOXLl5e4ceNK0qRJn7khauk00buOnJP4ceNI50b/feQMeLokCePLhAFt5M8RnSV18kRy8OQlKdVmmAwcs1AeBzL5EAAgcvhEZKV27dqZEPzXX39FShh+9OiRGRZPnjx5mDsuPH78WO7cuSPJkiWL1O26A9uocbt65SR5Ej9n7w4QrepWLCSvFsounYdNNzPqDRq7SBau3yuTBraRAjnS82wAAKJ/5HjXrl0yceJEqV69uhQvXvyZW1g9ffpUevToIUmSJJEsWbKYyUXmzZv33HWOHz9ufq7Wleg6uXPnNp0yXna77mLX4bOyYsshcwHeR29VdfbuAE6RIomf/D6sg8z8sp0kS5xAdh4+K8VafinDJy0zk4gAABCt4Thjxozy8OFDeVkjRoyQCRMmyKZNm8wIb+/evaVp06Zy6NChUGuc69SpY8o5/v33X7NO9+7d5Y033pCLFy9GeLvuRGstVdPqxSVLuhTO3h3AqZrWKCH7Zw2Q2uULmln2ev/wh5Rv/7UcO3uFZwYAEH3h+IMPPpCPPvrIXBn4Mn788UdTolG4cGGJGTOmdO3aVTJlyiS//vpriI8/evSoqXf+5JNPxM/vv3KC9957z5RW/PbbbxHerrs4deGa/L5yh/m6V8sazt4dwCWkTZFYFozsIr991koSJogr/+w9aVq+aR9w/RQJAIAoD8eDBw+W2bNnm1CaOnVqSZMmTZBbWOg002fOnJGyZcsGWV6uXDnTLzkk8eLFM//evXs3yGjy/fv3ZfPmzRHerrsYNX2l+ci4Rul8Ujh3RmfvDuAy9JqCtnXLmo4WVUrkNlOq6wQ51bt8K2cuXXf27gEAPP2CvG+++ealf7CWRagUKYKWBuh9LYcIidYPV6xY0dQTjx492jz2u+++M6UTGoojul0VEBBgbja6TRUYGGhuznbt1l0ZN3+D+brH21VdYp8AV5M+ZWJZ8t378ssf66Xv6PmyetsRKdj0cxnxUUNpU7uMR12YCwCQKMlDEQrHLVq0eOkfrOUOIR2UdqGIFStWqOvphXU6ct2tWzd58OCBNGnSRBo1aiSnT59+qe0OHTpUBg0aFOLFhwkSJBBnG79ktxkNy50xuSSUO7J9+3Zn7xLgskpmSSATP64tg6dukH0nr0qHIdNk4vy10vetspIicXxn7x4AIJLcu3dPXCIcR4b06f9ruXT58uUgy69cuWL/Xki0ddzIkSODLCtUqJAUK1bspbbbt29fc3Gf48ixXnhYpEgRp8+Qd//hI5nfb7b5+rMOb0qJEv8dK4DQad+cujUry7czVslnYxbJpgPnpdVXC+WHXk2lSbVijCIDgAfwt37S7xLhWEsU5syZI2fPnn1mlHb+/PkvXF8Dp14wt2LFCtNJQul2Vq1aZS74s7l586bpV6y1zUovsLGNDttGdvfu3SvDhw8P13aD8/X1NbfgfHx8zM2Zpi7ZYMoqsqZPIU2qFxcfn9BHwAH8n/7q9m7zmtQuX0haDZhgWr616D9B/ly7V37q85ZpCQcAcF8+UZDRInRB3tSpU02vYa3znTt3rqnnPXXqlPz555+mzVpY9e/fXyZNmiRjxowxAfedd94xyzt16mR/TK9evaRq1f/38/3888/N47VzxcKFC6VBgwamzKNmzZrh2q67CAx8IiOmrjBf93i7GsEYiID82dPJ5ol9ZGCH2uITK6aZPCR/k4GyYO0ezicA4OXD8bBhw2TGjBkmJKtx48bJ7t27pWfPnuH6qFKDrW5Dg2z9+vXN0PjatWslZcqU9sdoRwzHDhjaQu7AgQPy5ptvypdffmnu64Qk4d2uu/hjzS45eeGaJE+cwFyNDyBiYvvEkgEd6piQnD9bOrl644682eMnaTNwoty6c5/TCgAwYlgsFouEk44Oa7mDtlaLEyeO3Lp1S+LHjy/Xrl2TXLlyyY0bN8TdaaBOnDix6eXsrJpjfWpKtPpSdhw6a0a89A87gJf3MOCxDBizQL6essL8nmVInVTG929l2iQCALw7r0Vo5Fhbntl6DutFbgcPHrT3H9auEIgca7YfMcE4nm9s6dKkMqcViCRxfWPLV10byoZxvSRHxlRy/spNqfn+d9J52HS5e//lZ/8EALivCIVjR9pGTWt++/XrJ/Xq1TO1yIgcwycvM/+++2ZZLhwCosCrhbLL7un95P0mlcz9n+esNbPrrd91jPMNAF4qQuH4n3/+sX89ZMgQady4sZmhTmehGzt2bGTun9fac/ScLPvnoMSMGUO6v80bDiCqJIjnKz983FxW/vShZEqTzNT4V+wwQnp+O0cePHzEiQcALxOhmmNv4Oya4xb9x8u0JVulWY0SMuPLdtH+8wFv5H/3gXQfNVvG/7nR3M+bNa1MGthGSuTP4uxdAwC4cs0xotaZS9dl5vL/ZsDr1aoGpxuIJon84sm4/q1k4agukiZ5Ijl06pKUeecr+eyXBfLoMVO2A4A3iFA41nSuM8qVL19eChQo8MwNL2fU9JXy5MlTqVYyrxTNk4nTCUSz2uVfkf2zBphPbvR3cfC4xVKq9TDZd/wCzwUAeLgITSvStm1b06GiWbNmkiRJksjfKy92/dZdGTtvg/n6Y0aNAadJnsTPlDQ1qFJEOg2dJruPnpNiLYbI5x3rSs8W1ZmQBwA8VIRqjv38/MxEHJkzZxZP5aya4y/GLZb+vyyQwrkyys5pn4ZrUhUAUePKdX/pMGSqLFj334x6pQtmlUkD20quzP9Naw8A8PKaY90JW59jRB69Mv77WWvso8YEY8A1pE6eSOaP6CQTB7aRRAniyuZ9p6TwW4Pluxmr5OnTp87ePQBAJIpQOO7UqZN8/PHHcv8+U65GpkmL/pF/b96RzGmTS+NqxSJ12wBejr5ZbV27jKlFrl4qrzwIeCwfjvhdqnYaJacuXOP0AoA3l1WcOHFCSpQoYWbES5s27TMjnKdPnxZ3F91lFXrRT+6Gn8mJ8//K9z2bygfNqkT5zwQQMfrf5i9z15leyPcfPhK/+L4y8qPG0q5eOT7xAQA3z2sRuiCvdevWkilTJjMzHhfkRY55a3aZYJwscQJ5582ykbRVAFFBBwQ6NaooNUrnkzYDJ8qG3cdNTfK8NbtlbL8Wkj5VUk48AHjTyLHWGx8/flzSp08vnio6R471KdA2UdsOnpbP2r8hg96rG6U/D0Dkfuqjtcef/DRfAh4FSpKE8WX0x83krVolGUUGAG+5IE9DcaxYsSJlByCydsdRE4zj+saW95tU5pQAbiRWrJjSvUV12TWtnxTPl1lu3bkvLfr/Jo0+HmOuIQAAuJeYES2r6Natm0npeHnDJy83/75T91VJmTQhpxRwQzrV9D+/9ZbBHeuKT6yY8seaXZK/ySBTMgUA8PCyitSpU8vVq1clZsyYkiJFimc+Orx8+bK4u+gqq9h77LwUaj5YYsaMIUf/GCzZM6SMsp8FIHrsOnxWWg+caJ9Rr8VrpeT7Xk0laaIEPAUA4IkX5I0YMSJSfjhEvpmywpyGRlWLEowBD1EkTybZNrmvDBq7SL6atEymLtkiq7cfkfH9W0qtVws4e/cAAJEdjgMDA6VNmzYRWRUOzl6+ITOWbTVf92pZg3MDeBDfOLHlyy71pW6FQtJ6wEQ5evaKvNb1B+lQv7x882EjSZggrrN3EQAQWTXH7du3Z1aoSPDt9JUS+OSpVCmRW4rnyxIZmwTgYkoXzCa7pveTbs3/613+67z18krzz82FuAAADwnHefLkkV27uMjkZdz0vye/zttgvv64Vc2X2hYA1xY/bhz5tkdTWf1LdzMD5umL16XSeyPkoxG/m2njAQAeMH108+bNZebMmbJ37145fPhwkBte7Oc5a+XegwB5JWcGM5EAAM9XuXhu2TfzM2lfv5y5/+2MVVLk7S9ky/5Tzt41AMDLdKsI3p0iuAhs0qu6VTwMeCyZ6/SVqzfuyNTB78jbr5WK1O0DcH1LNu6XdwdPlkvXbptuNX1a1zKTAGmtMgDAzbpVnDt3LlJ+uLeavPgfE4wzpUkmTaoXd/buAHCC18oWkP2zBkjXb2bKtCVb5csJS2TRhn0yeVAbKZQrI88JALjTyLE3iKqRY51qNm/jAXLs7FX5tkcT6da8aqRtG4B7mrtqp3QcOk2u3borsX1iyYD2taV365ri48NMpADgFtNH2+zevVtmzJgh06dPN1/jxf5cu9sE46SJ4su7b5bllAGQhlWLyoHfB0i9SoXlceAT6ffzn/LqO8Pl8Gn3n1AJANxNhMLx9evXpUaNGlKkSBF55513pF27dubrmjVrmu8hZDpIrxMCqC6NK4lffPqcAvhPqmSJ5I+vO8qUz9tKYr94su3gaXOx3qhpK2mdCQCuHo67du1qhq+1nduDBw/k/v375utbt25Jt27dIn8vPcT6Xcdk64HT4hvHRz5oWtnZuwPAxejFzi1eL21qkWuWyWcu3u0+arZU7jhSTp7/19m7BwBeIUI1x0mSJJHt27dLjhw5giw/duyYlCxZUm7evCnuLipqWGp/OFoWb9gnHRtWkJ/7vh0p2wTgmfS/5rHz1kv3UXNM28cE8XxlxIeNpEOD8i/sGAQA3sLfVWqOHz16FOIO6LKAgIDI2C+Ps//4BROM9Y9ajxbVnb07AFyc/l/RoUEF2Tujv1QomtMEZL1or9YH38v5K+4/AAEAripC4bhcuXLSs2dPuXfvnn3Z3bt3pUePHlK+fPnI3D+P8c3UFebfhlWKSI6MqZy9OwDcRLYMKWXNL91lVPfGEtc3tizffFAKNB0kUxZv9oie8gDgEWUVhw4dklq1apka4/z585tlBw4cMOUWS5culbx584q7i8xheh3lyVr3Ewl88lS2TuorJfJnibT9BOA9tHtF6wETzLULSrtb/NL3bUmdPHInKgIAby6riHCf44cPH5rpozUU68d/+fLlk2bNmkncuOHrwPD06VPZsmWLXLlyRQoUKPBMHXNI9CLAbdu2mdrmTJkymU4ZEXlMdJ3snt/OkRFTV0ilYrlkzZgeL7UtAN4tMPCJDJ+8TAb+usi0fUuRxE9+7vOWNKpWzNm7BgDeFY41uO7fv998/cUXX0i/fv1e+ofrgegI9NmzZ0243rRpk3zwwQcybNiwUNdZu3atNGrUSNKlSydZsmQxwTpz5sxmxDpp0qRhfkx0nexbd+5Lxjf6yN37AfLXdx+YWbEA4GXtOXpOWg+cKHuOnjf336pVUn7o1UySJU7AyQXgNfydGY59fX3lzp07EidOHDNSHBm1bu+//76sWLFCtm7dag5sw4YNpmZ55cqVUrVqyDPHFStWzIwuz5o1y9zX0o6cOXOa9nL9+/cP82Oi62QPm7hU+o6eJwWyp5O9Mz/jKnMAkebR40D5fOwiGTpxqTx9apG0KRLLuH4t5fVyBTnLALyCfxSEY5+wPrBgwYLy7rvvmlZtavTo0c8NvS+i4XratGnSt29fc1C2C/10+1OnTg01HGs3DB0NttE652TJkpkOGuF5THTQHqXfzVhlvv64VU2CMYBIFSe2j3zRuZ7UqVDI1CIfOXNF3vhwtLSrV860fUvkF48zDgBRFY4nTpxoSinGjBkTKeH4/PnzZkRXyzWCh/DnTUU9atQo6dChg8SPH9+USixfvtwEXx0VDs9jgtNA7diGTt+JqMDAQHOLiEmLNsnl6/6SMXVSaVS1SIS3AwDPUyxPRtk2qY/0/2WhfD9rjYybv0FWbD4o4/q3NNc6AICnCoyCbBXmcKwhdv78+eZrLas4fPjwS/1gHf5WGlodJU+e3ITm0OiIcPbs2WX27NmSLVs2E6QbNmwYZCg9LI8JbujQoTJo0KBnluvMfwkShL+GTz/i/HL8IvN1/bI5Zc/uXeHeBgCER7NymSVX6loyZOp6OXP5hlTv8p00qZRXOtUtLnHjhPm/ewBwG45thSNLhP63/Pnnn1/6B2sNs60/siO9H1rHC+1s8frrr0vFihVNXbItZGsnCh8fH/n666/D9JiQaHlH9+7dg4wcZ8yY0awXkRqWP9fukbNX/SVJwngysEtTSZggfF08ACAiihcXaV63mvQePU/Gztsgv/99SHaduC7jP2spZQpm46QC8Cj+1k/6nR6OtaOEli3EjBmhOUQMba+mYVU7VTg6c+aMGe0NyYULF+T48eNBSjq0XrlGjRqyZs2aMD8mtLBuC+yOdB/1Fh5aT22b9KNzo0qSNLFfuNYHgJeh/+f8+mlLaVilqLw7eLIcO3dVKr03Unq1rCGD3qsjvnFic4IBeASfcGa0sIhQus2TJ48pN3gZGkT1ojstfbC5du2arF69Wt544w37Mu1VrC3YVOrUqSV27NhmEhJHej9DhgxhfkxU27jnhGzed0p84/jIB00rR8vPBIDgapbJL/tnDZCWr5c2pV5fTVomxVt+KbsOBx2UAAD8X4TidqdOnaR58+by+eefm/7E2t4teHgOC+1nrB0qWrZsKWXKlJFx48aZ2fXatGljf4xeALh582bTD1l/Tp8+fcyFgRqkdYRZL7bT72tvYxWWx0S14ZOWmX9bv1FG0qT4rxMHADhDkoTxZfLnbaV+5cLy3pfTZP+Ji1Ky9VDp3+4N6dv2NYntE4snBgBedoY8vSDvecKzyaNHj5pQrDPkaacKDd6OF8D9+uuvcuzYsSC1wtobedmyZXL9+nXTjaJ169aSNWvWINsNy2Oiom/ewZMXJX+TQf9dtDhnkOTKnDrM6wJAVPr35h3pNHS6zF2909wvljeTTB7UVvJlS8eJB+CW/F1l+mhtw/Y80VW+4Ion+51Bk2TCwk3SoHIRmft1xyjdRwAIL/0vf8aybdLlqxlmBk8t//qi05vy0VvVJFasiF9HAgBeHY69QURO9oWrNyVr3U/lceAT2Tyxj5QqEPaRagCIThf/vSXtBk+RJZv2m/tlC2WXiQPbSI6MqXgiAHh1OI7wMIH2ItYZ7gYPHmxfphfpaSs1b/XdjNUmGFcompNgDMClpUuZRBZ/976Zbtovvq+5kLhQ88Hy0+y/vfr/cQCI0Mjx/v37pXr16hIvXjw5deqUvcZYp5fW/sKtWrXyuncit+8+kIxv9JE79x7Kom/flzfKFYyW/QSAl3X64jV55/PJsmb7EXO/Wsm88ttnrSRjmqCTNAGAq3GZkWOdLKNjx45y8uTJZ6aNHjlypHijMXPXmWCcP1s6ee3V/M7eHQAIsyzpUsjKnz6U73s2lXi+sWXl1kNSoOkgmbhwU7gusAYArx051oR+7tw5k9C1K4NtEzqFn04HHRAQIN70TiTg0WNTa3zp2m1Ts9e6dplo208AiExHz1yR1gMnmF7tqk75V+TXT1vQlhKAS3KZkWOdGe/+/fvPtHXTlmtJkiQRbzNtyVYTjNOnSiLNa5Zw9u4AQIRp+8kN4z6WYe/XlzixfWTh+r2Sv+kg+X3Fds4qAK8QoXCsE3J88cUXZsTYFo4vXrwoXbp0kdq1a4s30QtXvp6y3HytrZD0jwkAuDNt6da7TS3ZPuUTKZwro9y4fU+a9h0rzfqOleu37jp79wDA9cLxiBEjZOXKlWZyDQ2HJUqUMDPR3bhxQ4YOHSreZNH6fXL49GVJ7BdP2tcr5+zdAYBIUzBHetkyqY981v4NE5hnrdhuapEXrtvDWQbgsSLc5/jhw4cye/Zs2b59uwnIRYsWlWbNmpkOFt5Uw1Lu3eGmBVKfNrVk6Pv1o3UfASC6bD94WloNmCiHTl0y99vWeVVG9WhiBgYAwFmYBMTFTvbG3celXLuvTSnF6YVfStoUiaNzFwEgWj0MeCz9f/lTRkxdacrqMqZOKhMGtJaqJfPyTADw7gvybFNIf/rpp9KgQQNz69evn1y4cEG8ia3WuNUbpQnGADxeXN/Y8nW3RrJubE/Jlj6FnLtyU6p1/lbe/2qG3Hvg/l2KACDCZRXLli2TN998U3LmzCnFixc3y7Zt2yYnTpyQBQsWmAlCPOWdyNpt+8XPL+Ez3z918Zo0+niM+Xru8PdMn9AUSfwkE03zAXiBu/cfSu8f/pCfZq8197NnSCmTBraRsoVzOHvXAHgR/ygYOY5QOM6bN6+pLx4wYECQ5YMGDZJZs2bJwYMHxVNOthRuIxIrTpjWiRvHR478MZiADMBrrNh8UN4ZPFnOX7lpuhf1bFFdPu9Y14wyA4DXhOP48eOb1m3BexrfunVL0qdPbyYD8cZwrHZM/VSK5skUpfsGAK7k1p378tHI32Xiwn/M/XzZ0srkQW2lWN7Mzt41AB7O31VqjnXkOKTR4QMHDkiePHkiY78AAG4iScL4MmFAG/lzRGdJnTyRHDx5SUq1GSYDxyyUx4FPnL17ABD14bhNmzbSqFEj+emnn0yt8datW83XjRs3lnfeeUcOHz5svwEAvEPdioVk/6wB0qR6MXny5KkMGrtISrUeKvuPe9fF2gDcW4TKKhynjH6RCLZRdjrKKgAg4mYt3yadv5phZtfTdpeDO9aVHi2qm8lEAMCVyyoiNNfxuXPnIuWHAwA8U9MaJaRC0VzSYcgUM5OodraYv3a36WiRM1NqZ+8eAERuOM6QIUNEVgMAeBGdGGnByC4yceEm6Tbid/ln70kp1HywDO/aUDo3rigxYzKKDMD18D8TACDKaBle27plTS1ylRK55UHAY/ng65lSvcu3cubSdc48AJdDOAYARDmdIGnFjx/K6I+bSfy4cWT1tiNSsNnn8tufG9322hQAnolwDACInj84MWNKlyaVZff0fvLqK9nlzr2H8u7gyVLnox/l0rXbPAsAXALhGAAQrfSCvHVje8rwrg1MJ4vFG/ZJ/iYDZeaybYwiA3A6wjEAINppS7derWrKTuusojf970vzT8dJ075j5dqtuzwjAJyGcByJ4sbxkRRJ/CJzkwDg0fJnTyebJ/aRgR1qi0+smDJ75Q4zirxg7R5n7xoALxWhSUC8qan02m37xc8vYZjW0WCsF50AAMJvx6Ez0nrARDlw8qK537p2Gfm2RxMzPTUARNckIITjaDzZAIDnexjwWAaMWSBfT1lh6o8zpE4q4/u3khql83HqAERLXqOsAgDgMuL6xpavujaUDeN6SY6MqeT8lZtS8/3vpNPQaXL3/kNn7x4AL0A4BgC4nFcLZTct395vUsnc/2XuOjO73vpdx5y9awA8HOEYAOCSEsTzlR8+bi4rf/rQXM9x8sI1qdhhhPT8do48ePjI2bsHwEMRjgEALq1qybyyb+Zn8u6bZU0d8oipK6RoiyGy7cBpZ+8aAA/k9Avyrly5ItOnTzf/FixYUJo2bSo+Pj7PXWffvn2yZMkSuXnzpmTKlMmskyzZs10i1q1bJ6tXr5b48eNLs2bNzGPDigvyAMD16IQh7QZPlsvX/U2v5L5takn/dm+YyUQAeB9/T7sg79ixYyYQL126VGLHji0DBgyQWrVqyZMnT0JdZ8yYMVK8eHE5ceKEOQlz5syRHDlyyJEjR+yPefr0qbRs2VIaNWokDx48MLe6devKoUOHounIAABR4Y1yBWX/rAHSvGYJefLkqXwx/i8p1XqY7Dt+wXz/7OUbsvPw2TDf9PEA4DIjxw0aNJBr167J33//LTFjxpSzZ8+aoDt+/HgTbkOSJ08eqVmzpnz33XfmvgZpXUdHj4cNG2aW6fc+/fRT2bNnj2TPnt0su3v3rty/f19SpUoVpn1j5BgAXJtOGKJdLK7fviexfWLJR29Vle9nrpaHjwLDNXnTkT8G06MecFP+njRy/PjxY/nrr7/krbfeMsFYadlDpUqVZP78+aGup+FWg67jdgICAiRNmjT2ZT/99JO8/fbb9mCs/Pz8whyMAQCur3G1YnLg94FSt0IheRz4RIZPXh6uYKz08UxXDcCR04q0dJRYQ61jgFV6f+PGjaGu99tvv0nHjh3ltddek8yZM8uWLVvMqHHnzp3t7yCOHj1qRo615GLHjh2SLl06adiwofk3NLoverPR7ajAwEBzAwC4nuSJ48ucr9rL1CVb5YPhM+VeBLpY8P884L4CoyCjOS0ca4mDSpgw6NTMOiRu+15INPgePnzYlFboSLPWGm/evNmUZ2j4tYXar7/+2ny/bNmysnLlSvnkk09kxYoVUrp06RC3O3ToUBk0aNAzy3ft2iUJEiR4yaMFAESlvKliyedtK0iPn1eGe91Dhw7K07tXo2S/AESte/fueU7N8enTpyVr1qym64RehGfToUMH2bZtmwmlwT169MiUT/To0cOMDCvd/VKlSknu3LllypQppuYkSZIkUqVKFVm1apV93TfeeEMePnwYZNmLRo4zZswo169fZ/poAHADeoFdqTZfhXu9LRN7S9E8Ye9mBMB1aF5Lnjx5pNYcO23kWEd1tQ5YR4Edw7Hez5cvX4jraLs3bd9WokQJ+7IYMWJIsWLFTKBWWpSdPn16s8yR3tfwHBpfX19zC07byr2otRwAwPki+n81/88D7ssnCjKa0y7I04vwtA544sSJZkRX7d2719QbN2nSxP64WbNmyVdf/TcSoKFXR4WXL18eZDR57dq1kj9/fvuy5s2by5o1a+wt4bS1m/Y7LlSoUDQeIQAAANyNU1u5Xb58WSpWrChx4sSRIkWKmO4VWv4wadIk+2PatWtnaor3799v7v/+++/Stm1bKVOmjGTLls0EYy3G1nZwWgZhG2KvXr266W+sNcY6qqzD7Vp7rOuEBa3cAMD9yiqKtRgS7vUKZE8nzWqUkDoVXpGCOdKbTyQBuIeoyGtOnyFPA6zWHdtmyCtXrlyQ72ug1RDdokUL+zJ97Pr16009sHasqFq1qplExJEGZr0A78yZM6aEQx8TUtlEaAjHAOAd4dhRpjTJpE75V0xQrlQsl/jGCfq3BYBr8chw7KoIxwDgHeH403dekz3HzsvKrYflYcBj+/IE8XylRum8JizrzHypkkXOH14Arp3XuNIMAODVGlQpKl90rif3Hz6S1dsOy8J1e2XRhn1y8d9bMm/NbnPTUotSBbKYoFy7POUXgCcjHAMAICLx48YxwVdv+qGqjkRrUF64fq/5evO+U+b26U9/Sua0yaV2uYKUXwAeiLKKUFBWAQDu5ezlG5K7Qf9wTSEdN46PHPljsKk1fp4LV2/K4g37TFAOXn7hF99XapTKJ7XLF6T8Aohm1By7+ckGAER9QL52626YH58iid8Lg3FwWn6xaushE5QXrd8nl67dtn/PsfyiToVCphMG3S+AqEM4jkaEYwDAi2gf/V1HzgUpv3BE+QUQtQjH0YhwDAAILy2/0NHkRRtCL7/QNnGvly1A9wsgEhCOoxHhGADwMii/AKIe4TgaEY4BAJFZfqElFzqqHFr5RZ3yBU2nDCYfAcKOcByNCMcAgKguv9CgvGob5RdARBGOoxHhGADgCuUXpQtmtU8+QvcLICjCcTQiHAMAnFV+YQvKwcsvsqSzTT5SSCoWzSm+cWLzJMGr+UdB610mAYnGkw0AQGSWX9Qsnd/e/SJl0oScXHgdf8Kxe59sAABepvxi5RZb+cVeuXzdP8TyC73lZ/IReAl/wrF7n2wAACK7/EInINGJSIKXX9jqlCm/gCfzJxy798kGACAqnL+i5Rd77eUXAY8C7d+j/AKezJ9w7N4nGwCAqHbvQYCs2no41PKLMgWzSe3yBSm/gEfwJxy798kGACC6yy92HLJ1vwi9/EJvFeh+ATfkTzh275MNAICrll8kTBBXapTKR/cLuBXCsZufbAAAXKn8QrtfLNqwL9TyC20Tp32V6X4BV0U4dvOTDQCAq5dfaPeL3UeDll9kTZ/iv8lHtPtFsVwSJ7aP0/YVcEQ4jkaEYwCAtzp3+YYs3rAv1PKLmqXzmTZxTD4CZyMcu/nJBgDAXcsvzEV9G/bJlVDKL3RUOV+2tGYZEF0Ix9GIcAwAQFCUX8DVEI7d/GQDAOBp5Re2C/pCK7/QEeXXyxWUFEn8nLqv8Ez+tHJz75MNAICnovwCzkA4dvOTDQCAt5RfbD94xowqh9b9wnHyEbpfIKIIx9GIcAwAQOSWX2hQXr2d8gtEHsJxNCIcAwAQNeUXK7T7xbq9snhj0O4XMWNau1+Uf8W0iqP7BV6EcByNCMcAAERP+YWZfGT9Xtlz9HyQ71N+gRchHEcjwjEAANHrrJZfaD/l9fueKb9IpN0vyuQ3M/XR/QI2hONoRDgGAMB57t5/KCu3Hn5h+YVOQJI3K5OPeCt/T2zltm7dOvnxxx/lypUrUrBgQfnkk08kbdq0oT4+MDBQJk6cKH/99ZfcvHlTMmXKJO3bt5dy5cqF+Php06bJDz/8IM2aNZMPP/wwzPtFOAYAwHXKL7Zp94tQyi+yafeLCq9I7XJ0v/A2/p4WjtesWSM1atSQ3r17S5kyZUyIPXr0qOzZs0cSJkwY4jrdu3eXKVOmyPDhwyVLliyydOlSGTFihKxYsUIqV64c5LFHjhyRatWqiR5io0aN5Ntvvw3zvhGOAQBw7fILDcqrtx2RR4+fLb8wk4+ULSDJmXzEo/l7WjguW7asZMyYUWbOnGnu379/34waf/bZZ9KjR48Q18mWLZu8/fbbMnjwYPuyAgUKmJA9cuRI+7KAgAApVaqUfPrppzJkyBCpVKkS4RgAAA8uv1i0Ya9cvXEnSPnFq69kN3XKlF94Jv8oCMcxxUk0CG/evFnq1KljXxY/fnwz0rty5cpQ1ytevLhs27ZNHj9+bO6fPXtWzp07JyVLlgzyOA3XhQsXlsaNG0fhUQAAAGfyix9X6lUqLOM/ayWXlg6XzRP7yKfvvCaFcmWQp08tsmH3cekzep7kbzJIctTrJx+OmCWrth4KMtoMOPIRJ9FAqzVE6dKlC7Jc769atSrU9SZMmCDvvvuupE+f3jz29OnTpsRCa4pt5s+fL0uWLJHdu3eHeX90pFlvju9EbDXOegMAAK6vWJ6M5jawQ21TfrF4w35ZvGGfrNlxVE5euCbfzVhtblp+UaN0XnmjXEF57dX8kjyxn7N3HREQFRnNaeHYNvLr6+sbZHm8ePHs3wuJXryntcpaKpE9e3ZZvny59O3bV4oVK2ZGlTV0d+jQQf78889Q65ZDMnToUBk0aNAzy3ft2iUJEiQI17EBAADXUCJLfCmRpZR83LiobD18UTbuP2duN+88lDmrdplbzBgxpGC2VFKuQEYpVzCjZE6dWGLEiOHsXUcY3Lt3TyKb02qOL168aEZ/Fy5cKLVr17Yv11HhAwcOmJKL4O7cuSPJkyeXn376Sdq1a2dfruvrKLR2sNDw3KdPH8mfP7/9+/v27TN1KJkzZ5aNGzdKrFixwjRyrPXQ169fj7QaFgAA4DrdL3RUWae13nf8wjPdL3REuXa5AlKucA6JE9tpY4l4Ac1rmg0js+bYac+2lkSkSZPG1A87huMtW7ZI+fLlQ1zn7t27ZlRZQ3Xwbe3du9d8rV0pdBTZUatWraRo0aKmlVtIwdg2gh18FFv5+PiYGwAA8BxlC+c0ty/fry9nLl03pRe27hdafvHDrDXmpuUXtV79r/vFa6/S/cLVREVGc2q3Cu0koT2Lt27dagLv3LlzzQV0GpBLlChhL3fYv3+/6VescubMaUaFZ82aZcKs1hxrVwoNwF9//XWIP0cvzKNbBQAACEv3ixVbDtknHwmp+4Vt8pE8WdJQfuGB3SqcOiSqLduOHTsmOXLkMCUM58+fl9GjR9uDsTpx4oTpe2yjoViDsI4Wa6DW9XXkeeDAgU46CgAA4EndL+pXLmJutvKLhev2yML1+2TvsfOm+4Xeev/wh2TPkNIE5drlC0qForkktk/In07DvTh9hjxb/bHOkKchOfhFdCdPnjTlFK+88op9me7yhQsXTD2wzpCXNGnS527fseY4rJgEBAAAONLyC9vkI2u2H31m8hHKL6Kfx00C4soIxwAAIKLlF2UL6eQjlF9ENcJxNCIcAwCAsNDyi60HTpugrKPKwbtf2MovtE65fJGclF9EIsJxNCIcAwCAiKD8IvoQjt38ZAMAAO9y556WXxw0I8raV/nfm8+WX/x3UV/Yu1/ozH/Xbt0N8z6kSOInmdIkE0/kT82xe59sAADgvZ480e4XL1d+ocE4d4P+8vBR2KdNjhvHR478MdgjA7I/4di9TzYAAIDN6YvXZNH6fbJow7PdLxL7xZNaZfKbNnGOk4/sPHxWirUYEu6TuGPqp1I0TyaPO/n+ntbnGAAAwFtlSZdC3m9a2dxCKr+YtWK7uTmWX2TLkNLZu+3xCMcAAABOljBBXGlQpai5afnF1gOnzKiyrfxi/a7j5oaoRzgGAABwIbFixZQyr2Q3tyFd6tnLLzQor95+RAIDnzh7Fz1aTGfvAAAAAF5cfrFsdDdZ/fNHnKooRjgGAABwEwni+Tp7Fzwe4RgAAACwIhwDAAAAVoRjAAAAwIpwDAAAAFgRjgEAAAArwjEAAICbSJHET+LGCd80Ffp4XQ9hwyQgAAAAbiJTmmRy5I/Bcu3W3TCvo8FY10PYEI4BAADciAZdwm7UoawCAAAAsCIcAwAAAIRjAAAAIChGjgEAAAArwjEAAABgRTgGAAAArAjHAAAAgBXhGAAAALAiHAMAAABWhGMAAADAinAMAAAAWBGOAQAAACvCMQAAAOAq4XjUqFGSOXNmiRs3rpQoUUI2bNjw3MffuHFD2rVrJ+nSpZN48eJJ7ty5zTYcrV27VurUqSPJkyeX1KlTS8OGDeXYsWNRfCQAAABwd04Nx+PHj5dPP/1UfvzxR7lw4YJUqVJFatWqJWfPng11nfbt28vmzZtl1apVJigPGzZMevfuLVOnTjXff/LkiQwYMEA6duxoAvGOHTvMsmrVqsndu3ej8egAAADgbmJYLBaLs354njx5pEaNGvL999+b+7orGTNmlJYtW8rQoUNDXCdr1qzSunVrGThwoH1ZoUKFTLAOPoJso2FbR6dXrFhhQnJY+Pv7S+LEieX27duSKFGiCB0fAAAAok5U5DWnjRzrqO+RI0ekUqVK9mUxYsQw9zdt2hTqes2bN5e5c+eadR8+fCgLFiyQU6dOSaNGjUJd599//zX/6skDAAAAQuMjTnL58mXzb8qUKYMsT5UqlWzbti3U9QYPHmzCsI46qzhx4siYMWOkbNmyIT4+MDBQunfvLkWKFJFixYqFut2AgABzc3wnYltfbwAAAHAtUZHRnBaOQ6OlFTqCHJpWrVrJwYMHZe/evZI9e3ZZvny5vPXWW+Ln5/fM6LFuSy/e01FmvdAvZszQB8q1jGPQoEHPLN+1a5ckSJDgJY8KAAAAke3evXueU3OsZRXaTUJLJBo0aGBf3qJFCzl37pzpOBHctWvXzEjznDlzTAcKG61RPn36tKxfv96+TA+rQ4cOpuxizZo1ki9fvufuT0gjx1r/fP36dWqOAQAAXJDmNc2TkVlz7LSR42TJkpk2bBpcbeFYA+3ff/9twm5IYsWKZf4NPrKs69m+Z7uv3So0GK9evfqFwVj5+vqaW3A+Pj7mBgAAANcSFRnNqa3cevbsKb/99pssXrzYjCT36dNHbt26ZYKtjZZFFChQwHydNGlSc8Gelj/s379f7t+/L/PnzzcjyY6jz126dDHLNRjnz5/fKccGAAAA9+PUIVENvjoc3qlTJ7ly5YoULFhQli5datquhWbWrFnSt29f0w9ZSx70sUOGDJEPPvjAXnrx888/m69todpm7Nix5mcCAAAALtfn2JXR5xgAAMC1eVSfYwAAAMDVEI4BAAAAK8IxAAAAYEU4BgAAAKwIxwAAAIAV4RgAAACwIhwDAAAAVoRjAAAAwIpwDAAAAFgRjgEAAAArwjEAAABgRTgGAAAArAjHAAAAgBXhGAAAALAiHAMAAABWhGMAAADAinAMAAAAWBGOAQAAACvCMQAAAGBFOAYAAACsCMcAAACAFeEYAAAAsCIcAwAAAFaEYwAAAMCKcAwAAABYEY4BAAAAK8IxAAAAYEU4BgAAAKwIxwAAAIAV4RgAAACwIhwDAAAArhSOz507J9u3bxd/f/8wPd5iscjJkydlx44dcvXq1UjbLgAAALybU8Pxw4cPpWHDhpI7d25p2bKlpEmTRn744YfnrrN3717Jly+fvPrqq9KhQwfJmjWr2caDBw9earsAAACAU8PxoEGDZOvWrXLixAk5dOiQTJ8+Xbp27SpbtmwJdZ1OnTpJ5syZ5fz582bkWNdbtWqV/Pjjjy+1XQAAAMCp4XjChAnSrl07SZs2rblfr149KVCggFkemn///VdKlCghPj4+5n6mTJkkQ4YMZvnLbBcAAAD4L2E6wcWLF+XKlStSrFixIMtLliwpu3btCnW9zz//XHr27ClZsmQxI8jLly+X+/fvS+fOnV9quwEBAeZmc/v2bfPvjRs3JDAwMMLHCQAAgKhhu65Mr0dz+3CsoVMlT548yHK9b/teSKpUqWKCbt++fc2IsV6Y16dPHzOC/DLbHTp0qCnHCE5rmgEAAOC6rl+/LokTJ3bvcBw7dmz7xXOO9MK6OHHihLiOvit47bXX7DXH+jjtSKFhWUd3+/XrF6HtKg3b3bt3t9+/deuW+Tlnz56NtJPtLu/AMmbMaM5rokSJxFtw3Dzf3oDXOa9zb8Dr3Lte57dv3zYDpMmSJYu0bTotHGsAixkzply4cCHIcr1vGwUOTksmdu7cKV988YU96Op26tatKwsWLDDhOCLbVb6+vuYWnAZjb3qR2egxc9zeg+fbu/B8exeeb+/irc93zJgx3f+CvPjx45t2bBpqbe7duycrV66U6tWr25cdP37cXius7wr04HXU2JGOcqZMmTJc2wUAAABcZuRY6QiwBlYtaShTpozpRZwqVSrTv9hm2LBhsnnzZtm/f7/EixdP2rdvL5988ok8efJEsmXLZi7IW7p0qSxevDhc2wUAAABcKhxXrFhR1qxZY3oUa1/iggULypQpU8TPz8/+mJw5c8qjR4/s9/WxpUqVkmXLlsncuXNNXfCmTZukdOnS4drui2iJxYABA0IstfBkHDfPtzfgdc7r3BvwOud17g18oyCvxbBEZu8LAAAAwI05dRIQAAAAwJUQjgEAAAArwjEAAADgChfkOZOWWh88eNBMHpI/f37x8fGJknVcTUSPQdfRiVG0TZ47ishx67TkR48eNW0C06dPL95y3I8fP5YjR46Yixt0hkhvep2r06dPm3aRBQoUkCRJkognH/f27dufmTBJZx7NkiWLeMPzfefOHTl27Ji5sDv4rKqedtw6McbevXtD/F7evHnd6vgj8nzr61xn1NV1tNNVeC7Qd/fj1pa42gZXn+cYMWKIOzoYzgzy0nnN4oWOHDliyZMnjyVVqlSWjBkzWtKnT2/ZtGlTpK/jaiJyDNOmTbOUKFHCkjRpUouvr6/FHYX3uC9fvmxp27atJXHixJbChQtbkiVLZilTpozlxIkTFk9/vgcOHGgeX6hQIUuGDBnMOgsXLrS4k5f5Xb1y5YolXbp0epGyZcmSJRZPP+7MmTNbcuTIYSlbtqz9Nnr0aIunH/eTJ08sffv2tcSPH9+81vU89OzZ0+LJx713794gz7Pe9LnX1/ratWstnvx8T5w40fw/nitXLkuBAgUsCRIksAwdOtTiTiJy3GPGjDF/x/Lly2fJlCmTWf/QoUMWdzItAhkkMvKaV4bjIkWKWOrUqWMJDAw099977z3zB/HBgweRuo6ricgx9O/f37J582bL2LFj3TYch/e4t23bZpkwYYLl8ePH5v7du3ctVapUMQHZk49bH/fVV1+Z41VPnz61fPzxxyZAPHz40OIuIvq7qsdbq1YtS+/evd0yHEfkuDUU6u+2O4vIcffr18+SPHlyExhtz/2PP/5ocSeR8TepVatWlmzZspnj99TjvnnzpiVWrFiWr7/+2r5s5syZ5nfc9vx76t+xmDFjWmbMmGHu63P84YcfmqCsbw7dRf8IZJDI+N3wunC8c+dO80uhJ9vm3LlzlhgxYljmzZsXaeu4mpc9BncNx5H13OnIg/5HYwvM3nLcf/31l9mOjqZ7+nHrG4Pq1atbLl686HbhOKLHreF42LBh5g+puzzHL3vct27dssSLF88ycuRIi7uKjN/v27dvmze+X375pcWTj1s/8dN1NmzYEGQdXbZ69WqLpx738OHDzWi5o6NHj7rdJwXhzSCR9bfP6y7Is01FXbRo0SA1dmnTprV/LzLWcTWecAzOPO5t27aZukR3qb99meM+deqUrF+/XmbOnCm9evWSrl27SurUqcWTj1snCxo1apRMnDjRLWvyXub5HjJkiLRr106yZ88uFSpUMHWZnnzcOmnUgwcPpE6dOnLx4kXzOK3H9bb/12bMmGEm2GrTpo148nFrfbHOjtu9e3eZP3++/PXXX/Luu+9K7dq1zYRhnnrcyZIlk7t37wZ5bV+4cMH8u2PHDvFUuyLpb77XheMbN25IokSJJHbs2EGW68UI+r3IWsfVeMIxOOu4//77bxkzZoz0799fvOG4Fy5cKL179zZ/TPQijrfffls8+bj1j0fz5s3lp59+knTp0ok7iujzrbNKXb9+XXbv3i1nzpwxyxo3bixPnjwRTz1uDcT6BkhnUC1evLgJh/rmT1/z3vT/2vjx401A1NDg6cfdtm1buXfvnnmzr7dDhw5Jly5dzP9vnnrcDRs2lFSpUkn9+vVl0aJF5s3Q+++/L/HixeNvfhi4xysjEumLK/jV2UpHEuLEiRNp67gaTzgGZxy3vsOuV6+edOvWzfwH6w3HrSPFOrqmowz6H2yVKlVMoPDU4+7Tp48JxdqVZMOGDWYUWR04cED2798vnvx862va9gdX/9B++eWXsnPnTtOlxZP/P9eSwkuXLsnZs2dlz549smrVKhk5cqTMmjVLvOH/tX379plPw9q3by/uJCLHrW/6KleuLO+9957pTKK/16NHjzZvDLZs2SKeetzaaUe70RQrVsy8Efzjjz9k3LhxEjduXBOQPVXsSMo6XheO9aNx/Sjp2rVr9mU6SnLlyhXJlClTpK3jajzhGKL7uDUkVK9e3QSIb775RtxJZDzfOrrWs2dPM+Kybt068dTjTpgwoQlLGpL1pmUGasKECWZ0zZt+v23lM7aPXz3xuG1t6vSjdVuZlLaHeuWVV0w5kTc83/q6zpgxo9SqVUvcSUSOe+XKlaY9ZadOnezL6tata9pzLl68WDz5+dbf5+HDh8uSJUtk9uzZZiT55s2bpk2lp8ocWVnH4mWuX79uiRMnjmX8+PH2ZcuXLzcF3AcOHLAv27p1q+XUqVPhWsfTjtsTLsiL6HHv2rXLXMzQtWtXizuKyHHbulQ42r17t1lH1/WG17m6dOmS212QF1nPt7Zx0wtPz58/b/HU49bOK0mSJDEdaWz0Qltt96StDD39da7Hr506BgwYYHE3ETnuP//803z/7Nmz9u/fu3fP4ufnZ/n+++8tnvx8379/P8h2tF1hmjRp3KrLVlgySFTkNa8Lx7bWIPqfo55s7aGn/Vy1pY0j/Y+yR48e4VrHE4/78OHDlvXr15v2VvqC06/15u/vb/HU49YeifrHo0KFCvbjtd0CAgIsnnrcS5cutVSrVs3y22+/WVasWGH5+eefLVmzZjXnwdYSx1Nf5+4ejiNy3PocV6pUyfwR0edeg6F2cejVq5fFnUTk+da2balTp7aMGzfOPM+NGzc2v/N6Vbunv861jZm+ATpz5ozFHYX3uDUg5s2b11KsWDHLnDlzLAsWLDBdabQHrjt1aInI862tKfW1ruHwo48+ssSNG9eybNkyizs5HIYMEhV5zT0uvY9kgwYNMjN/zZ0718yeohceaaG6o5IlS5rHhGcdTzxuLeLXj6VUiRIlzMfO6tdff5V8+fKJJx63zpCWJ08e81GM7Xht5s2bZ2pTPfG4a9asaerUtJxg+vTpkiZNGnPBll6QFytWLPHk17kjrUsrW7asJE2aVNxJeI+7WrVq5iIf/YhdX/P6Mfuff/5pSoncSUSe786dO5sL0aZOnWrKhnQGLa09dqeZMCP6OtdZ8rTEwF3L6cJ73Fpfu3HjRvn+++9l8uTJZp0iRYqYzjTu0oUnos/3pEmTZOjQoebvVs6cOU0Nsr7W3cmMMGSQqMhrMTQhR+qRAAAAAG7K6y7IAwAAAEJDOAYAAACsCMcAAACAFeEYAAAAsCIcAwAAAFaEYwAAAMCKcAwAAABYEY4BIIK0yfyFCxdc5vzdvHlTZs6cKY8fP3b2rgCA2yIcA0AEtW7dWrZt2+Yy5+/EiRPSvHlzM+ubq7p+/boJ8DpzFQC4IsIxAHiIZMmSSdOmTc0U2K7q2LFjJsA/fPjQ2bsCACHyCXkxACAiNPRt2rRJHjx4IAULFpRMmTIF+f7ixYvlzp07EjNmTEmfPr0UKVJE4sePH+QxOrJauXJlMwK8Z88eyZ07t6RIkUJWr15twu+hQ4fk5MmTkitXLnOzSZo0qdSrV09ix45t7l+9evWF69js3r1bzp07J3ny5JE0adKY/dRtxY0b95nH2rbbpEkT2bFjh5w5c0YqVaokCRIkkD///NM8RvchW7ZsUrhwYYkRI4ZZdvfuXVm5cqW9JMXX11eyZMkipUuXNsuuXbsmW7ZsMT+zaNGi5ngAILoRjgEgkmzYsEEaN25sAnHKlClNSO7YsaN8+eWX9sesWLFCLl++LE+fPpUjR46YOuEFCxaYEGmjI6uvvfaaHD582ITnFi1amECqy6dPny4XL14021+zZo0MGTJEevToEaSsQreZJEkSOXjw4AvXUe3atTOBvFy5cnL8+HETnpcsWSKXLl0yQTk423anTp1qaq718fpGQAP//PnzzWMCAgJk+/bt5lwsXbpUEiZMaML+2rVrzfcXLlwoPj4+Ur58eROOR48eLf369ZMSJUqYc7Nr1y4ZO3asNGzYkNcngOhlAQBESIIECSzz5s0zX/v7+1tSpEhhmTJliv37p0+ftiROnNiyYsWKULfx0UcfWSpVqhRkmf7XXLZsWcuDBw/sy9asWWOWDxw40L5s4sSJlnjx4lkCAgLM/W3btpnH3Lx5M8zrLF++3OLj42PZtWuXua/LK1eubNa7dOlSiPts226PHj2ee350W6VKlQry8//55x+z7p07d+zLNm7caEmUKJHl4MGD9mXz5883y65fv/7cnwEAkY2RYwCIBDoSqiOjWiowe/Zs2+CDKRvQ0dpq1arZH6vlDTpq7O/vL35+frJ169ZnttehQ4cQSxo6depk/1pLGbR84+zZs5IjR45Q9+1568yZM8fsm23kWuuVu3XrZvb5Rbp27Rrich311e3rz8mQIUOIx+do4sSJZvT5wIEDsn//fnPe9KYlKlq2Ub169RfuCwBEFsIxAESC06dPmzIBraV1pDW8mTNnNl9ruYB2uNC63JIlS5oL6LQE4v79+yZYa82uTdq0aUP8ObqOjQZx9aKL2563jtYZa4B3FPx+aILv47///muCrP5bqFAhSZQokbkA70UXCOq50/OgQd1R/fr1JV68eGHaFwCILIRjAIgEGgQ1/Gp9r9behkQvRvvjjz9MbbCtlnfRokVm+X/VFP9nu4gtqmlwvnXrVpBlGlTDIvg+jho1yoRvvUBP3yionj17yt9///3Cc6cjx1r3DADORis3AIgENWrUMGUEepGao0ePHpkuDEovxNML5VKnTm3/fvDR0uhWtmxZE8519NrG1nEivPT4smfPbg/GOhmJXmzoSMtIgo9216pVS1atWmVGmR3pRYhMaAIgujFyDACRQEc+Bw8eLO3btzddGrSsQMsFNPyOGzfOtGKrUqWKaePWqlUr06pN26FpyzRneuedd+Tbb7+VqlWrSps2bUzLtxkzZkRo9PrNN9803Tpy5sxpSi4mT55sSix0ZNhG27tpizbtTFGxYkXJmjWr+bnz5s0zQf2DDz4w6+7du9d0zNBWdrbWdAAQHRg5BoAIatSokbngzOaTTz4xJQRaY7t+/XpTL6ttzDT0KX2s9vHVlmra0kwDtLZ20z7EjgFQ7wdvoZYqVSqz3LFkQ7evy3Q0OqRJQMKyjpZBaMs5HfnWfdPey3qBnOMob3AhbdcWjnXU+cqVK+YNgrax++2336RmzZr2x2hPZx0l1v3Qixj1gjsdadavf/rpJ9NyTi/g01ptvbAveA9oAIhqMbRlRZT/FACAy7px40aQi/Z0VFcvLNRRZADwNpRVAICX00lAtK2bzsSnI74TJkwwJREA4I0YOQYAL6d10OPHjzc9hrWcQ8tFHGfsAwBvQjgGAAAArLggDwAAALAiHAMAAABWhGMAAADAinAMAAAAWBGOAQAAACvCMQAAAGBFOAYAAACsCMcAAACAFeEYAAAAkP/8D89IrvcKizEEAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 800x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 6. Hyperparameter Tuning: Learning Rate Sweep\n",
    "learning_rates = [0.01, 0.1, 0.2, 0.3, 0.6, 0.9]\n",
    "performances = []\n",
    "hidden_nodes_baseline = 100\n",
    "\n",
    "print(\"Starting Learning Rate sweep. This may take a few minutes...\")\n",
    "\n",
    "for lr in learning_rates:\n",
    "    print(f\"Training network with Learning Rate: {lr}...\")\n",
    "    # Initialize a fresh network for each test\n",
    "    testANN = ann(784, hidden_nodes_baseline, 10)\n",
    "    \n",
    "    # Train 1 epoch\n",
    "    for record in list: \n",
    "        values = record.split(\",\")\n",
    "        data = np.asarray(values[1:], dtype=float) / 255.0 * 0.99 + 0.01\n",
    "        target = np.zeros(10) + 0.01\n",
    "        target[int(values[0])] = 0.99\n",
    "        testANN.backpropagation(data, target, lr)\n",
    "        \n",
    "    # Evaluate on the Test Set\n",
    "    score = 0\n",
    "    for record in list2:\n",
    "        values = record.split(\",\")\n",
    "        correct_label = int(values[0])\n",
    "        data = np.asarray(values[1:], dtype=float) / 255.0 * 0.99 + 0.01\n",
    "        \n",
    "        outputs = testANN.feedforward(data)\n",
    "        if np.argmax(outputs) == correct_label:\n",
    "            score += 1\n",
    "            \n",
    "    # Calculate performance (accuracy as a decimal between 0 and 1)\n",
    "    performance = score / len(list2)\n",
    "    performances.append(performance)\n",
    "    print(f\"Performance for LR {lr}: {performance:.4f}\\n\")\n",
    "\n",
    "# Plotting the exact graph requested\n",
    "plt.figure(figsize=(8, 5))\n",
    "plt.plot(learning_rates, performances, marker='s', markersize=8, color='#003366', linewidth=1.5)\n",
    "\n",
    "# Formatting to match the requested style\n",
    "plt.title(\"Performance vs. Learning Rate\")\n",
    "plt.xlabel(\"learning rate\")\n",
    "plt.ylabel(\"performance\")\n",
    "\n",
    "# Setting axes limits and ticks\n",
    "plt.xlim(0, 1)\n",
    "plt.xticks(np.arange(0, 1.1, 0.1))\n",
    "plt.ylim(0.8, 0.98)\n",
    "plt.yticks(np.arange(0.8, 1.0, 0.02))\n",
    "\n",
    "# Adding horizontal grid lines\n",
    "plt.grid(axis='y', linestyle='-', alpha=0.7)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bda99b0",
   "metadata": {},
   "source": [
    "## 6.2 Hyperparameter Tuning: Hidden Nodes Capacity\n",
    "In this experiment, we evaluate the effect of the network's capacity by varying the number of hidden nodes `[10, 50, 100, 200, 500]`. The Learning Rate is kept constant at $0.2$.\n",
    "\n",
    "The resulting curve demonstrates the law of diminishing returns in neural network architecture. While increasing the number of nodes initially provides a massive boost in classification performance, the accuracy plateaus after approximately 200 nodes. Beyond this threshold, adding more nodes significantly increases computational cost and memory footprint without yielding proportional accuracy gains."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "0ead825a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting Hidden Nodes sweep. This will take a while...\n",
      "Training network with 10 hidden nodes...\n",
      "Performance for 10 nodes: 0.8064\n",
      "\n",
      "Training network with 50 hidden nodes...\n",
      "Performance for 50 nodes: 0.9183\n",
      "\n",
      "Training network with 100 hidden nodes...\n",
      "Performance for 100 nodes: 0.9274\n",
      "\n",
      "Training network with 200 hidden nodes...\n",
      "Performance for 200 nodes: 0.9344\n",
      "\n",
      "Training network with 500 hidden nodes...\n",
      "Performance for 500 nodes: 0.9167\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsoAAAHFCAYAAADrKN8IAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjExLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlcelbwAAAAlwSFlzAAAPYQAAD2EBqD+naQAARi9JREFUeJzt3Qd4VFX6x/E3ISGEAAm9hl6UIh0EREClKEVQEPnDsggIsgu6Ii7L2hZFQVdxEV1sgAURFRFQEAWWKk0wKCDF0HsnCSEJhMz/eQ/OmEluwmQyYUq+n+cZZ+bOvTd37sjNL2fec06QzWazCQAAAAAnwc5PAQAAABCUAQAAgCzQogwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAvhqUjx8/LmvXrpW4uDiXtzl8+LBs3rxZ4uPjc7UOAAAA4HNBWUNs7969pWHDhtK2bVuJiYm57jbJycly//33S506deRPf/qTlCtXTqZOnZrjdQAAAACfDcrbt2+Xvn37ysaNG13eZvz48bJp0ybZu3ev7Ny5U2bPni2PPvqo0z5cWQcAAADw2aA8aNAg6dOnj4SGhrq8zcyZM2Xo0KFSvnx587xnz55Sv359szwn6wAAAADZCRE/cuzYMTl58qQ0bdrUaXmLFi0cZRuurGMlJSXF3OzS0tLk3LlzUrJkSQkKCvL4ewEAAEDu2Gw2SUhIkAoVKkhwcHD+DsoaXJWG1/T0uf01V9axMnHiRFOyAQAAAP+iAzhUqlQpfwdle4mGdtZLLykpSQoWLOjyOlbGjRsno0ePdjzXETgqV64s+/fvl2LFinn0fQAAACD3dGSzatWqSdGiRSUv+FVQjo6ONs3qR48edVquzzXUurqOlbCwMHPLqESJEgRlAAAAHxQSci3K5lWZrE+Mo5yd2NhYR21x4cKFpXXr1rJw4ULH64mJibJs2TLp2LGjy+sAAAAAPt2ifOrUKdmzZ4+cPn3aPN+2bZv5y0Bbfu2tv5MmTZINGzaYoeTUhAkTTODVUolWrVqZ8ZHLlCkjw4YNc+zXlXUAAAAAn21R3rJli/zjH/+Q1157Tdq0aSOfffaZea6tv3a1atWSJk2aOJ63a9dOVqxYIQcPHpQpU6ZIvXr1zKx+RYoUydE6AAAAQHaCbDquBiyLwyMjI02nPjrzAQAA5L+85vM1ygAAAIA3EJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALAQIj7g8uXLEh8fLyVLlpSgoCCXtrly5YokJCRIiRIlMr124cIFuXjxotOy0NBQKVu2rMeOGQAAAIHNqy3KaWlp8sQTT0hUVJRUrVpVKlWqJF999VW228TGxkrHjh2lWLFiZps6derI6tWrndb5xz/+IbVr15Zbb73Vcevfv38evxsAAAAEEq8G5ddee01mzpwp69atMy3KY8eOlb59+8rOnTst17969ap0795dChUqJKdPnzbbjB49Wrp27SrHjh1zWveee+6RI0eOOG7Lli27Qe8KAAAAgcCrQfmtt96SoUOHSqNGjSQ4OFgeffRRqVy5srz77ruW6+/Zs0d27dol//znP6VIkSJm2fDhw035xYwZMzKtf+7cOVPWAQAAAPhNUD516pQcPHhQ2rRp47T8tttuk02bNlluEx4ebu7T1x9rK/OlS5dkw4YNTutqCUe1atVMoG7durX89NNPefI+AAAAEJi81plPSydUqVKlnJbrcy3FsKI1ye3atTN1zW+++aZZd8qUKaYEQ4O3XbNmzUxLc+PGjSUuLk7++te/mrrmHTt2SLly5Sz3nZKSYm52uk+VmppqbgAAAPAteZ3RvBaUtdTC6g3qaBYFChTIcjttKX7hhRfksccek6SkJHnggQekd+/ecuDAAcc6Ws5hFxkZKe+9956ULl1a5s6dKyNHjrTc78SJE2X8+PGZlsfExEhERIRb7xEAAAB5JzExMTCDcsWKFc39iRMnnJafPHnS8ZqV4sWLy+TJk52WNWzYUJo2bZrlNlqyUb58eacwndG4ceNMx8D0LcrR0dGmVVpH2AAAAIBvsVcABFxQ1vCpnfiWLl1qRrqwty4vX75cRo0a5Vjv/PnzpkOefQxkHVLO3hptb/H95Zdf5JVXXnEss9lsTuMxHz9+3NRDV69ePcvjCQsLM7eMQkJCzA0AAAC+Ja8zmlcT4DPPPGNCcvPmzaVVq1by6quvmuUjRoxwrPPkk0+ajnrbt283z59//nnTOtyhQwfZvXu3GSljwIAB0rlzZ/O61hm3b9/ebFevXj05dOiQaS3WMZp1PQAAAMDnh4e77777ZNasWfLhhx9Kr169TPP5qlWrTD2xnQ79lr4D3uOPP2465d17773y0ksvmecffPCB43VtFZ42bZqpR+7Zs6c899xzcuedd8qWLVsooQAAAIDLgmxap4BMNLRrR0AdNYMaZQAAgPyX17zaogwAAAD4KoIyAAAAYIGgDAAAAFggKAPwmBfeXyTBzYebewAA/B0DBAPwCA3Hz7690Dy23z8ztCtnFwDgt2hRBuDRkGynz2lZBgD4M4IyAI+HZDvCMgDAn1F6AcAtySlXTBD+98ffZ7uernMh4ZI8PeQeKVq4kISEFOCMAwD8AhOOZIEJRxDodK6hhMRkOZ9wyQTZ8/GXzGO9N8/tjy/+8dqFhCQ5H59oHqdcTnXr5xYuVFCKRRSSYhHhUqxIIYksEv7Hc3Of7nGRrJcVDOXvfADI7+LzeMIRftPAL7/qf+6dhTJ+eI9831ksNfWqXLiY5BR0nUNv4rVwaxl6L0la2o2fmPNS8mVzO3E2Plf7KRQWmk3Azjp4/xHMry0PKxgiQUFBHnt/AIDAQVCGXwnEkRWSki+bsGtaak2YtXjsaOm1h95E85q2COeWBsXiRQtLVNHCUrxY4T8e//48/eP067331Rp5cca3Lv+c5x7uJqP6dpD4xGSJT0z6/T5Z4i9eexxn7tMttz++6Lx+YlKKo/RDb6fOJeTq/YeGFMi69Tp92LYI2em305ZyAjcABBaCMvxGViMreDsspy9hyFS2kEUJQ/rX3C1hSK9oRCGJKhIuxYtF/B5mw68feotFmG3CCxV062dO+EtPCSsYmmVHvvSef+SP1v+SUUUkt63oF5NSHCH7WsDOEL71sQnY6Zc7h/OESynms7uSelXOxiWaW24EBwdlbsU2Ldv28hLXSkuKFA6T4GD6WQOALyAoIyBGVshtWLaXMDiXKmT12PMlDBqynAJtkfQtuBkDsPNjDbve6iBnP+fZheX0IdkT9L3qudJbbqSlpcnFSylZtnBfL3inD+j6+etN/1/Qmyf+8LGH50inMpLrl5bYl9FxEgByj6AMvw7Jdvq6tgyO6N3OUbaQsV7XuaX3xpQwZA691o/9uRUxu7Ds6ZDsSXq+TbAsEi4ixd3ej7ZKa811liE7wzLLFvDfW8ZTr6aZfer/j3o7Khdy9R4jwsMsW7ivF7LTt4BraKfjJID8ilEvssCoF/4Tkj3J1RIGe0uuJ0oYAvWz8uWQ7Is0cGsZjsshO1NpyR+PtXbbk1ztOJlpBBM6TgLIY4x6gXzl9PkE2b73mOzYe8zcvzNvdY73USIyIkclDPbaXW+WMAQCeyhmRBL3aEdADaR6K1Mid0McXb6SalqkLUN2VqUlXu44eb2QTcdJAN5Ai3IWaFHOW1rysGPfcdkee1R27LsWivW5BuXcGD+8uzz7cDePHSeQn2ntfsKl5BzXb2cM554obcqq42TmgO36mNz+XPIE4BpalOHX9Bfrr/uPO1qJ7aH4+Jm4LFvVqlUoKfVrVJR6NcpLveoVZN3Pe+W/c1dd92fxVT/gWfoNi/nmpVjEDe84GecI4M6vebLjpF5vihYOy35owCyCtwnov2+nHScLFCBwA4GIznz5wI2YoEO/ot25//i1VuK9R2XHXn18TA6dOJflNpXLlTBBuH6NClJPb9XLy83VypsOSOn1v7ullCsVeUNHVgDgnx0n/6jjznpoQHvHSd2ffXluudpx0tECnkVpiZanAPAdBOUA5+kJOrROcffBE7+XShyT7bHX7vcfO2t+6VgpXyryWhhOF4rrViv/+y/NwB5ZAYDnaAuwBlK96XXF0x0nXR+TO3PHSW0s0FtW35bltuPkHwHbtdISZpwEPIOgHMByM0GHdgb67dCpa63D+447OtfFHjmV5ZjBpYsXNa3CjlBcs6J5ntuvbbMLy4RkAN7uOOnq0IBWHSft4VxbyvOq42RkDoYGzDjde3hYaL6fcfJGfCsL30VQzucTdGhnnb1HTqfrUKe1xMdNq7F9TNeMdISIjC3EGohz+wvHFYysAMCX6BjTpaKKmNuN7DiZ1ZTv9o6TnppxUmuvXRka0Cpkp18eEV7QLztOevpbWfgfRr0IwFEvXB17uFzJYtlOoaw9wh1hOF0o1q88tUUGAOBbXOk4aVnHnUXHSU9xtePktdbvrFu9b2THyax+l/JNpm9h1Avk2QQdJ87Gm3v9aq1u9WsjTKQvm4guW5xADAB+JK87TjqFbB/rOJl+RklXSkuy6zjp6reyCHy0KAdYi3Jw8+GSRZ86S9ownLpxml9+JQYA8G32jpOZSkVy0HHSvm1W3356ouNkZLoykgPHzsrPvx257va0LPsGWpSRI9rZICdTPuv6hGQAQF53nCxbMneNTimXr0iClpXkoOPkHzNS/vGapzpO0rKcP9CiHGAtyjkpv+CvYQBAfpNdx8kH//l+jval38qm/fhOnh0rro8WZeRYdmMO2xGSAQD5UXYzTu45dCrH38oisFGYGsBhuV2TWpavEZIBALD+3am/I13B79L8gaAcoK5eTZO9R89kWs4/bAAAcheW+V2afxCUA9TSjb/KkZPnpURkhDz7cFdTR8U/bAAAcheW+V2avzAzX4CasXCduR9wd0tTQ0UdFQAAuevvQ0jOfwjKAejMhYsyf+VW83hwj9bePhwAAPw6LD/3zkLT4MQkI/kPQTkAzV6ySa6kXpUmN1WWhrWjvX04AAD4LQ3HBOT8ixrlAKOzIE1f8IN5POTeNt4+HAAAAL9FUA4wP+06JL/8dkTCCoZIv87NvX04AAAAfougHGBmLLzWmnxfh8aWg6kDAADAT4Ly1KlTpUaNGlKkSBFp1aqVrF+/Ptv1T58+LQ8//LBUrlzZTC3dqVMn2blzZ673GwiSki/L7CU/mseDe1B2AQAA4LdBeebMmTJ27Fh57bXXZO/evdK6dWsTfA8fPpxl/e29994r27dvl6VLl8r+/fulWbNm0qFDBzl//rzb+w0UX63cKhcSLkmV8iXljuZ1vH04AAAAfs2rQfnf//63DB48WHr27Clly5aVV1991bQST5s2zXL9ffv2mZbhSZMmSZ06daRkyZLy4osvSlBQkMyYMcPt/QZa2cVD3VtJcLDXvywAAADwa15LU9oCrCUT2hpsp4FXn69bd22yjIzS0tLMffoQqNvo87Vr17q930Bw4NgZWb5pl3mvg7ozdjIAAIDfjqN8/Phxc1+mTBmn5aVLl5bNmzdbbqM1xw0aNJBnnnlGPvroIylVqpRMnjzZ7Mu+P3f2q1JSUszNLj4+3tynpqaam6+b8fuQcHc0qyMVS0f6xTEDAADkRl7nHZ+bcERbh7UWOavXFi5cKKNHj5ZGjRpJUlKS9OnTR3r37n3d+uPs9qsmTpwo48ePz7Q8JiZGIiJ8e/SIq2lp8t68Vebx7fXKZvsHAQAAQKBITEwMzKBcrlw5xygW6Z06dcrUFWelatWqMm/ePKdlLVu2lJo1a+Zqv+PGjTMBPH2LcnR0tDRu3NjUN/uyZRt3yonziRJVNFxGP9RLCoWFevuQAAAA8py9AiDggnKJEiWkdu3asmrVKrnvvvvMMm3xXblypfTv39/l/eioFtqC+sQTT+Rqv2FhYeaWUUhIiLn5sg8XbTT3/bu0lCIR4d4+HAAAgBsirzOaV4dG0HA7ffp0+e677yQuLk6efvppOXfunDzyyCOOdYYPH27KLOy0JlnX13ribdu2mdKLO+64w5Rf5GS/geJcXKJ8tTLGPB7MlNUAAAAe49Wm0mHDhsmFCxfkoYceMqUR9evXl8WLF5vyCrurV686FWrff//9MmrUKOnVq5dERkZKv379ZMKECU4jYbiy30Axe8kmSbmcKg1rV5LGdaK9fTgAAAABI8iWXQ83H6BDwukhFihQ4IbXvGgQ1xZpX65RbtJ/gsTsPixvjOkrox68w9uHAwAAEDB5zbeLbzOMmQxnMbsOmZBcMDRE/q9LC04PAACAB5FC/djMr69NoNKzfUMpGVXE24cDAAAQUAjKfio55YrM+vbaaBeDe7Tx9uEAAAAEHIKyn1qwaqucj78k0WWLy10tbvb24QAAAAQcgrKfmrHwWtnFoO6tpUABPkYAAABPI2H5oUMnzsnSjTvN40HdWnn7cAAAAAISQdkPffD1OjNkXodmdaR6pdLePhwAAICARFD2MzqutH20iyHMxAcAAJBnCMp+ZuWWPXLg2FmJLBIu93Vo7O3DAQAACFgEZT8zY+EP5r5f5+YSXqigtw8HAAAgYBGU/ciFhEvy5f9izGPGTgYAAMhbBGU/8ul3P5qJRhrUrCjN6lbx9uEAAAAENIKyH5ZdDO7RWoKCgrx9OAAAAAGNoOwnfvntiGz+9aCEhhSQ/ne39PbhAAAABDyCsp+1Jve4vaGULl7U24cDAAAQ8AjKfiDl8hWZtXijeczYyQAAADcGQdkPfL36FzkblygVy0RJp1vrevtwAAAA8gWCsh+YsfDaTHx/7tpKChTgIwMAALgRSF0+7sjJ8/Ldhh3m8UM9Wnv7cAAAAPINgrKP+/Cb9ZKWZpN2TWpLzegy3j4cAACAfIOg7MPS0tKcxk4GAADAjUNQ9mFrYmJl39EzUjSikNx/ZxNvHw4AAEC+QlD2YfbW5Ac7NZOI8DBvHw4AAEC+QlD2UXEXk+SLZVvM48E92nj7cAAAAPIdgrKP+uz7HyUp5YrUrV5eWtav5u3DAQAAyHcIyj4+drK2JgcFBXn7cAAAAPIdgrIP2rH3mGzcvl9CCgTLgLtbevtwAAAA8iWCsg934uvW9hYpW7KYtw8HAAAgXyIo+5jLV1Ll48UbzeMh99KJDwAAwFsIyj5m0dptcvp8gpQrWUy6tKrn7cMBAADItwjKPlp28edurSQkpIC3DwcAACDfIij7kGOnL8jiH7abxw91Z8pqAAAAbyIo+5CPFm2QtDSb3NaoptSpWs7bhwMAAJCvEZR9hM1mc5RdDO5BazIAAIC3EZR9xA8/75XfDp2SiPAw6XNXU28fDgAAQL5HUPYR0xesNfd9OzaTIoULeftwAAAA8j2Csg9ISEyWz5duMY8ZOxkAAMA3eD0oL1iwQO68806pX7++9OvXT2JjY7NdPzk5WSZNmiS33367NGjQQLp27SoLFy50WufZZ5+VmjVrOt3uuece8VWfL90sl5IvS50qZaXVLdW9fTgAAAAQkRBvnoVvvvlGevfuLf/+97+lVatW8vrrr0vbtm1lx44dUqJECcttHn30UVm8eLFMmzZNqlatKkuWLJFevXrJ119/7QjDp06dklq1asnUqVMd24WFhYmvmrFwnbkf3KONBAUFeftwAAAA4O2g/K9//UsGDBggf/vb38zzjz76SMqXL29C8FNPPWW5zdKlS2XIkCHSvXt381xblWfNmiXLli1zajWOiIgwLcm+buf+47Lul71SoECwDOx6q7cPBwAAAN4uvUhISJCffvpJOnfu7FhWsGBBueuuu2TlypVZbte+fXtZsWKFXLx40Tzfvn277Nu3Tzp06OC03tq1a+WWW24xLdT//Oc/zc/zRTN/b03u2qaBlCsV6e3DAQAAgLdblI8cOWLGDi5XznliDX3+yy+/ZLndu+++KwMHDpQyZcqY8oxz587JW2+95WhhVrp87Nix0q5dOzl69Kg8/fTTpsxj8+bNJoxbSUlJMTe7+Ph4c5+ammpueeFK6lX5aNF68/jP3Vrm2c8BAAAIRKl5nJ28FpTT0tLMfWhoqNNyDbJXr17NcrtnnnlGfvjhB/nkk0+kRo0a8v3335u6ZX2sHfzUCy+8IAUKFDCPmzRpIo0bN5Zq1arJnDlzTMi2MnHiRBk/fnym5TExMaaMIy+s+eWQnDyXIMWLFpIyYSkmyAMAAMA1iYmJEpBBuVSpUub+7NmzTsvPnDnjeC2juLg40/Hvgw8+MB34lJZXrF+/3oRjrV9W9pBsV6lSJdPxTzsJZmXcuHEyevRopxbl6OhoE7KLFSsmeeGlz64F44d6tJFbW7bIk58BAAAQqOJ/rwDwyaCsZQ1aH6x1wDlVtmxZE0Q15Pbo0cOxXFuLu3TpYrmNlkZoS3Tx4sWdluvz48ePZ/mzdLsTJ05IVFRUluvoqBhWI2OEhISYm6edOBMni9ddC+5De7bNk58BAAAQyELyOD+51ZlP64I1zGpLrb3cQXXr1k3WrFnj8n4eeeQRef/992XXrl2O+mMN3g8//LBjHe2IZx/NQuuStQVZh5HT1mWl9cxffvmldOzY0Ty/fPmyPPnkk46W6kuXLslf/vIXU87Rt29f8RUfL94gV6+mmXGTb65W3tuHAwAAgAzciuFjxowxtcQHDx6UKlWqOJY//vjjMmHCBPnuu+9c2o92uNNOfQ0bNjR1wFoyoUO96ZBvdjom8qFDhxzPP//8cxk2bJhpkdZOe+fPn5dBgwY5hpPTmmcN8BqotfVZX9f9L1++XKpX943JPLQTY/qxkwEAAOB7gmya2nJIR6bYsmWLVKxY0UyQYd+FtvLqOMjaipvTQmxtpdZtMzahnz592szGp2Ua6SUlJZlt9Fgy1iTbnTx5UiIjI6VQoUJu1bzotvqePF2jvO7nvdJmyCtSuFBBOb7kFSlWJNyj+wcAAMgP4vMwr7ndoqwHU7RoUfM4/UxyutydWhFtTc5qZInSpUtbLg8PDzdBPTva6uyLZiz8wdw/0LEpIRkAAMBHuVWj3KJFC/niiy+cgrKWOWjZRZs2lBJk5+KlZPls6bXRLii7AAAA8F1utSi//PLL0qlTJzNDnpZdaOc5Hc84NjY2R5358qO5y3+Si5dSpGZ0Gbmtke9PsQ0AAJBfudWifOutt8qGDRvMcGra8W7JkiXSqFEj2bRpk5ngA1mbvuBa2cXgHq2dylYAAADgW9wefK5u3boyffp0zx5NgNtz8KSs3RorwcFBMrDrrd4+HAAAAHi6RVnHJN66dWum5bosu+mn87uZX18bEu7u1vWlYhnnSVMAAAAQAEF54sSJZpKPjHSZ1i8js9TUq/LhN+sdZRcAAADwbW4F5bfffltGjBhhOdOezq6HzJas3yHHz8RJqagi0q3tLZwiAACAQAzKOtudVUc0XaYz6SHrsZP/dE9LKRiat/OSAwAAwEtBuWXLlvKf//wn0/LXX3/djLEMZ6fOxcvXq38xjxk7GQAAwD+41bT54osvyh133GHGTL799tvNWMqrV682nfn+97//ef4o/dysxRsl9WqatKhXVerXzH42QQAAAPhxi3KrVq3MmMm1atWSRYsWybfffiu1a9c2y/Q1/EH/iJj+e9kFrckAAAD+w+1iWZ1o5MMPP/Ts0QSgTTsOyK/7jkt4WKg82Lm5tw8HAAAALspVrzIdMzkhISHT8qioqNzsNqDM+H0mvt53NpXIIuHePhwAAADkZenFrl27pG3btlKoUCEpXrx4phuuuZR8WT79/kfzmLGTAQAA8kGL8tChQ00gXrx4McE4G3OXbZGExGSpXrGU3N6kltsfEgAAAPwkKMfExMihQ4ekZMmSnj+iADJj4bUpqx/q3lqCg91qvAcAAICXuJXeoqOjJTk52fNHE0BiD5+SVT/tMZOwDOrOlNUAAAD5IiiPGjVKHn/8cYmLi/P8EQWID76+1prcuVVdqVSWum0AAIB8UXrxwgsvyMmTJ+XLL7+UUqVKZZrO+sSJE5KfXb2aJh98s948ZuxkAACAfBSUX331Vc8fSQD5fsOvcvTUBSkZGSE9br/F24cDAACAGxWUBwwY4M5m+caM32fiG3BPSwkrGOrtwwEAAIAbGIrBw85cuCgLVv3sGO0CAAAA+WxmvnXr1sncuXPNMHGpqalOr82fP1/yq1mLN8iV1KvS9ObK0rB2tLcPBwAAADeyRXnWrFnSsWNHOXXqlKND3/79+2XBggVmtr78ymazOcZOphMfAABAPgzKkyZNkk8//dQEZvX+++/L1q1bZcyYMZlGwMhPtuw8KNtij0qhsFD5vy4tvH04AAAAuNGlF7GxsaZFWYWGhsqlS5ekcOHCMnbsWKldu7bkV/bW5Ps6NJaoooW9fTgAAAC40S3KKSkpEh4ebh5XrFhRfv31V/P44sWLcuXKFcmPkpIvy+wlm8zjwT3oxAcAAJBvO/PZ9e7d2wwXp/fffPONo6U5v5m3IkbiLiZJ1QolpUOzOt4+HAAAAHgjKK9ff23WOfXiiy+aDny67LbbbpPx48dLfh47WYeECw5m1D0AAAB/F2TToRqQSXx8vERGRkpcXJwUK1YsyzP0wvuL5Ll3FoqeRe3IuH/hi1KlfEnOKAAAgI/kNa+VXuRnGpKffXuh43n1iiUJyQAAAAHCraCsqV2HiFu7dq2cP38+0+vbt2+X/BaS1d4jZ8zyZ4Z29dpxAQAAwItB+aGHHjIjXTz44IMSFRUl+Y1VSLazLycsAwAA5MOg/P3338uOHTukSpUqkt9kF5LtCMsAAAD+z63hGbRo2j6Ocn7iSki20/V0fQAAAOSjoDxixAj5+9//bmbky090dIu8XB8AAAB+XnrRr18/ad68ucyePVvKly9vhkVL78CBAy7va/PmzTJt2jQ5efKkNGjQQMaMGSMlS2Y9vFpaWpp8/vnnsnjxYtORsHLlyjJkyBBp0qRJrvbrivHDe7jcomxfHwAAAPkoKP/5z382AVVn5MtNZ75169ZJhw4dTAt1p06d5L///a+0adNGtmzZIhEREZbbPP300yYAP//881K1alVZsmSJ3HrrrbJixQqzrbv7dYW9g54rYfn5R3rQoQ8AACC/TTii9cmxsbFSsWLFXP3w9u3bS4kSJWTevHnmeUJCglSoUEEmTJggjz32mOU2NWrUMKNt6IyAdvXq1ZMuXbrIa6+95vZ+czKA9fVqlQnJAAAA/j/hiFs1yhqQCxQokKsfnJSUJGvWrJGePXs6lhUtWlTuvPNOM6pGVrSMYtu2baYEQ504cUKOHTsmDRs2zNV+c9qyrGHYCiEZAAAgn5deaMvsu+++a1K8Ow4fPmzCbqVKlZyW63Mto8jKhx9+KMOGDZNq1apJdHS07N6925RhDBw4MFf7TUlJMbf0f6Go1NRUc8to3KDO5uf8691vHMv+NaybWW61PgAAADwrrzOXW0H5zTfflFOnTsncuXOlVKlSmTrzaSvv9Vy+fNncZxxmrnDhwo7XrHz88ceyfPlyeeqpp0wZhrYSaxmGlltoa7O7+504caKMHz8+0/KYmJgs65o731JajnZtLO8vipGhXRub59qJEAAAAHkvMTHR94KyvRY4N4oXL27uz50757T87NmzjtesToaOXvGf//xHHnnkEbOsR48e8ttvv5ngvHDhQrf2q8aNGyejR492alHWFuvGjRtnW/PSrFkz+e8zLr1lAAAAeJC9AsCngrI2cw8aNChXP1jrnEuXLi0//fSTdO16bTQJpc916DkrWqit5RE62kV6+nzr1q1u71eFhYWZW0YhISHmBgAAAN+S1xnNrc58Dz/8sKMzXW5orfP06dNNGYfSsZE18Opyu8mTJ5ufp3TkCp02e+bMmY6alOPHj8s333wjrVq1ytF+AQAAgOy4FcNvuukmU7vbtGlTyQ2tCd6xY4fUqlXL1Bvv3LlTXnnlFcd4yOrXX3+VDRs2OJ7PmTNH+vfvb1qRdSxnHQHjtttuMx36crJfAAAAwOPjKOsEHlonrOG0bt26UrBgwUxBOif27NljZtC7+eabTefA9DTkasmFTipip63JOo6z1h1rC3PGES5c2a+3x+UDAABA7uR1XnMrKGcc5SIjN3bpcwjKAAAA+TuvuVV6oWMVAwAAAIHMraCcVakDAAAAEChyNaaGjiShNcRaaqG1yo0aNfLckQEAAAD+FpS1E12/fv1k6dKlUqhQIVOznJSUJJ06dZLZs2dLyZIlPX+kAAAAwA3k1jjKjz76qCma1iHiNCBfunTJPL5w4YI89thjnj9KAAAA4AZza9SLqKgo2bx5s9SsWdNpuU4l3aJFCzl//rz4O0a9AAAAyN95za0W5cuXL1sejC7TKaYBAAAAf+dWUNaZ8MaMGSOJiYmOZRcvXpQnnnhC2rZt68njAwAAAPynM9+UKVOkS5cuUqFCBalXr55ZplNGa0nGkiVLPH2MAAAAgH/UKKvk5GSZM2eOCcg66oUOD/fggw+aUTACATXKAAAAvs1nZuarX7++bN++3TyeMGGCPP300zJo0CCPHxAAAADgVy3KYWFhkpCQIAULFjQtyG42RPsNWpQBAAB8m8+0KDdo0ECGDBlihn9Tb775Zpbrjhw50jNHBwAAAPh6i7KWXWi5RWxsrKlLrlOnTpbr7tq1S/wdLcoAAAD5O6+51ZmP0gsAAAB4m09OODJt2jSPHwgAAADgS9wKyqNGjZK0tDTPHw0AAADgz0H5pptukpiYGM8fDQAAAODPM/ONGDFC+vXrJ88//7yZaESHjMsYpAEAAAB/5nZnvuwEwhjLjHoBAADg23xmHOX0Dh8+7PEDAQAAAHyJW0G5UqVKnj8SAAAAwN8786kLFy7IJ598Ii+88IJjmXbwYzQMAAAA5NsaZZ2lr2PHjhIeHi779+931CTrFNft2rWTgQMHir+jRhkAAMC3+eSEI6NHj5ZHHnlE9u3b57R85MiRMnnyZE8dGwAAAOBfLcqa3LVDnyb39NNZJyYmSokSJSQlJUX8HS3KAAAAvs0nW5SDg4Pl0qVLmYaK++233yQqKspzRwcAAAB4iVtBuUuXLjJhwgTTkmwPyseOHZO//vWv0q1bN08fIwAAAOAfQfm1116TZcuWSZUqVcwoF82bN5fq1avLuXPnZOLEiZ4/SgAAAMAfapRVcnKyfPHFF7J582YTlps0aSIPPvigGQkjEFCjDAAAkL/zmttBOdARlAEAAHybT05hrY4cOSLTpk2TnTt3mud169aVESNGSMWKFT15fAAAAID/1Ch/9913UrNmTVm4cKFJ8XqbP3++WbZ06VLPHyUAAABwg7lVenHzzTebeuTnnnvOafn48ePls88+k19//VX8HaUXAAAAvs0na5QLFy5shoPLOGbyhQsXTOmFTjzi7wjKAAAAvs0nJxzRFmWrVuMdO3bITTfd5InjAgAAALzKrc58gwYNkt69e8vTTz9txlDWRmkdJk4nIXnqqadk165djnWvF5wPHTokH374oZw8eVIaNGhg9h0WFpbl+iNHjjRD02XUtm1b+fOf/2wez5o1S1auXOn0urZ0a2kIAAAAkGdB+dFHHzX3OhOfVZBNL7vKDm2VbtOmjbRv315uvfVWeeONN+Sjjz4yITc0NNRyGw3mV65ccRp9QwNwy5YtHcvWrl0rW7dulUceecSxrESJEjl8lwAAAMjP3ArKhw8f9sgPHzt2rJmoZN68eWYq7IEDB0q1atXk448/lsGDB1tuY281tnv++eelSJEipnNhelWrVpWhQ4d65DgBAACQ/7gVlCtVqpTrH3z58mUzzNxbb71lQrIqX768dOjQQb7++ussg3LG1uqZM2eakFy0aFGn13bv3m1avrXAW8syOnXqlOtjBgAAQP7h9oQjuaW1yVpCoS2/6WmL8po1a1zax/Lly+XAgQPy8MMPOy3X4F29enWz76NHj8r9999vbh988EGW+0pJSTG39L0oVWpqqrkBAADAt+R1RvNaUE5KSjL3GVuC9bn9teuZPn263HLLLdKiRQun5c8++6xpnbbTkHzbbbeZlucuXbpY7mvixImWnf1iYmIkIiLCpeMBAADAjZPXQxJ7LSjbx7o7f/680/Jz5865NA6ervfVV1/Jq6++mum19CFZtW7dWqKjo2XdunVZBuVx48bJ6NGjnVqUdZvGjRvnybh8AAAAyB17BUDABWUNoRpAdeSLu+++22ks5vr16193ex0CTkssBgwY4NLP01bq7Ebg0CHprIalCwkJMTcAAAD4lrzOaG5NOOKRHxwcLH379pUZM2Y4ms1//PFH2bBhg/Tr18+xno6xnHGqbKXb9enTJ9PsgFqrsmjRIqdl7733npw+fdopkAMAAAA+GZTVSy+9ZP4SaNiwodx3331y1113yYgRI5wCrXbs+/LLL52208lNfv7550yd+JS2Mmvtsu5TA7eO0/z444/L66+/bkowAAAAAFcE2bKrR7gBdOSLFStWOGbma9SokdPrOnnIqVOnTJC205Csnex0Fr+s6PBwOulI8eLFzVjNpUqV8qm5wwEAAJA7eZ3XvB6UfRVBGQAAIH/nNa+WXgAAAAC+iqAMAAAAWCAoAwAAABYIygAAAIAFgjIAAABggaAMAAAAWCAoAwAAABYIygAAAIAFgjIAAABggaAMAAAAWCAoAwAAABYIygAAAIAFgjIAAABggaAMAAAAWCAoAwAAABYIygAAAIAFgjIAAABggaAMAAAAWCAoAwAAABYIygAAAIAFgjIAAABggaAMAAAAWCAoAwAAABYIygAAAIAFgjIAAABggaAMAAAAWCAoAwAAABYIygAAAIAFgjIAAABggaAMAAAAWCAoAwAAABYIygAAAIAFgjIAAABggaAMAAAAWCAoAwAAABYIygAAAICFEPGyhIQEmT9/vpw8eVIaNGggnTt3znb9V155RS5fvpxpeePGjaVr165u7xcAAADwmRblI0eOmBA7depU2bt3rzz00EPSp08fsdlsWW6TkpIiycnJjtvRo0flmWeekd27d+dqvwAAAEB6QTYvpsf+/fvLnj17ZN26dRIaGmrCbr169WTOnDnSu3dvl/YxefJkGTdunAnMpUqV8th+4+PjJTIyUuLi4qRYsWK5ep8AAADwvLzOa15rUb569aopjRg4cKAJs6pOnTrStm1bmTt3rsv7mTFjhvTq1csRkj21XwAAAORvXqtRPnTokFy6dElq167ttFyfb9y40aV9bNiwQXbs2CFTpkzJ9X61pENv6f9CUampqeYGAAAA35LXGc1rQfnixYvmXpvL04uKinK8dj3Tp0+X6tWryx133JHr/U6cOFHGjx+faXlMTIxERES4dDwAAAC4cRITEwMzKNvDp73l1k5rTFwJpnpiPvvsM1OfHBQUlOv96n5Gjx7teK7bR0dHm9E0qFEGAADwPRnzXsAE5cqVK0t4eLjExsZKp06dHMt/++03U1N8PZ9//rkkJSXJoEGDPLLfsLAwc8soJCTE3AAAAOBb8jqjBXvzjXXv3l0+/vhjR32JDuW2evVque+++xzrLVq0SN59913Lsotu3bpJ+fLl3dovAAAA4LPDwx04cEDatGlj6oybN29uRqVo2LChLFiwQIKDr2X4oUOHmk5727dvd2ynw73ddNNNJkTfc889bu33ehgeDgAAwLfldV7zak1B1apVTQD+4osvzAx6b7zxhvTo0cMpzGqrsYbc9M6cOSPPPfecdOnSxe39AgAAAD7bouzLaFEGAADwbQE74QgAAADgywjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAAEEZAAAAcA0tygAAAICFEPGyq1evyrp16+TkyZPSoEEDqVOnjkvbnT17VtavXy+FCxeW2267TQoWLOh4bdOmTbJnzx6n9aOioqRbt24eP34AAAAEJq8G5fPnz0vnzp3lxIkTUq9ePVm7dq0MHz5cXn311Wy3mzJlijz11FPSsmVLE5SffPJJWbBggVSqVMm8PmPGDPn222+lbdu2jm2io6MJygAAAPCPoKxhNz4+Xnbs2CFFixY1LcvaOqzhuWPHjpbbzJ8/X0aPHi1LlixxrLN7925JSUlxWq958+Yya9asG/I+AAAAEHi8VqNss9nk008/lSFDhpiQrFq3bi0tWrSQTz75JMvtJk6cKL169XIK0lquUaNGDaf1zp07J/PmzZPly5ebxwAAAIBftCgfPnxYLly4IPXr13darnXKP/30k+U2SUlJsnnzZhOuY2Nj5eeff5YKFSqY1uOQEOe3oq3UH3zwgRw9etTUK7/++usydOjQLI9HW6TTt0prS7dKTU01NwAAAPiWvM5oXgvKcXFx5r548eJOy0uUKOF4zaoDX1pamqk/fvnll6Vhw4YSExNj6pQXL14sVapUMev93//9n6ljDgsLM8+nTp0qI0aMMIFat8mqpXr8+PGZluv+IyIicv1+AQAA4FmJiYmSl4JsWgPhBdoiXKtWLVm6dKncddddjuV/+ctfZM2aNbJt27ZM25w5c0ZKly5tWp11ZItChQrJ5cuXTclG1apVZe7cuVn+PN1uzJgxMnbsWJdblLUDoIbzYsWK5fr9AgAAwLM0r5UsWdI0suZFXvNai7KG0NDQUDl48KDTcn1evXp1y21KlSplhnnTzn4akpUOC3f33Xdft+Oetgpr0M6Ktj7bW6DT05KOjGUdAAAA8L68zmhe68ynofTOO++Uzz//3LHs9OnT8r///U+6du3qWLZhwwb55ptvHM+7d+8uv/76q9O+9Lm97EJLM3S4ufR+/PFHOXTokBlODgAAAPDp0gv1yy+/SJs2bcz4xq1atTLjH2sr8w8//OCYQEQ74GlY3r59u6PFWQOvtirrths3bjSjZGgJh46bfOXKFVOHrOUcOjazBuT//ve/0r59e/nyyy8lODjY5ab8yMjIPGvKBwAAQO7kdV7z6hTWt9xyi2zdulWqVatmRrAYNGiQrFq1ymmWPQ3Q2opspy3Huk3NmjVNnbI+37lzp2NyEQ3aW7ZsMaNp6Hr6d8Ds2bPlq6++cjkkAwAAAF5tUfZltCgDAAD4toBuUQYAAAB8FUEZAAAAsEBQBgAAACwQlAEAAAALBGUAAADAAkEZAAAAsEBQBgAAACwQlAEAAAALBGUAAACAoAwAAAC4hhZlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAALBAUAYAAAAsEJQBAAAACwRlAAAAwAJBGQAAAPDVoHz48GHZvHmzxMfHu7zN1atXZfv27bJv3z6P7hcAAADwelBOTk6W+++/X+rUqSN/+tOfpFy5cjJ16tTrbjdv3jypVKmS3HvvvebWsWNHOXPmTK73CwAAANiFiBeNHz9eNm3aJHv37pXy5cvL/PnzpVevXtKiRQtp2bKl5TarV6+WPn36yHvvvSeDBw82y1auXCknT56UUqVKub1fAAAAIL0gm81mEy/Rlt4RI0bIc88951jWoEEDadOmjbz99tuW29xxxx0SGhoq3333nUf3m5GWa0RGRkpcXJwUK1YsR+8LAAAAeS+v85rXWpSPHTtmWoGbNm3qtFxbfWNiYiy3SUlJkbVr18rkyZMlISFB9uzZIxUqVDCtxrnZr33ferPTE67OnTsnqampbr9PAAAA5A17P7S8avf1WlDWAKpKlizptFyf21/LSOuQr1y5Ir/88oupP9aW49jYWGnVqpXMnj3baduc7FdNnDjRlGxkVK1aNbfeHwAAAG6Ms2fPmpblgAnKWj5h73iXXlJSkhQsWDDbbZYuXSpbt26VMmXKmBOjQXnMmDEyc+ZMt/arxo0bJ6NHj3Y8v3DhglSpUkUOHTqUJyceufvrMTo62oxqQlmMb+Gz8W18Pr6Lz8Z38dn4Nq0AqFy5spQoUSJP9u+1oKxBJzg4WI4ePeq0XJ/rG7ainfUiIiLkvvvuMyHZ3lKsnfs+++wzt/erwsLCzC0jDcmEMd+knwufjW/is/FtfD6+i8/Gd/HZ+DbNfnmyX/GSwoULS+vWrWXhwoWOZYmJibJs2TIz3JudllbYa4v1JOhrGUPwkSNHpHTp0jnaLwAAAOCzw8NNmDDBhFcte9DyCR3rWFuKhw0b5lhn0qRJsmHDBjO5iHr++efN6BU6ooXeb9y40dQnf/755znaLwAAAOCzE460a9dOVqxYIQcPHpQpU6ZIvXr1zKgWRYoUcaxTq1YtadKkidMwb+vWrTOtyK+88ors379fVq1aZcZJzsl+r0fLMDSMW5VjwLv4bHwXn41v4/PxXXw2vovPJn9/Pl4dRxkAAADwVV5tUQYAAAB8FUEZAAAAsEBQBgAAAHxt1AtfpZOY7Nu3z4zJrLP/4cbQcnkdClCHAWzUqJHlOjoz444dO0zR/k033SRBQUFurYOc0XO6e/duM465TsST1XiVe/fuNZP13HzzzWaoRnfXQc6cOnXKMVZ8xllJc3Jd49qXd3bu3GnOb/PmzTN1OuK6duP9+uuvmWbrLV68uOn8nxHXNe/Rc68TyOnvC6vfOzfkuqad+fCHZ5991hYWFmarW7euuR8yZIjt6tWrnKI8NnnyZFvt2rVtUVFRtoYNG1qus2rVKlu5cuVsVapUsZUqVcpWv3592759+3K8DlyXlJRk+/vf/24rUaKEOZcVKlQwn9MPP/zgtN65c+ds7dq1sxUtWtRWq1YtW7FixWxz5szJ8TrImZiYGHNO9XNp3LixLTw83PbAAw/YLl26lOPrGte+vPPzzz/bChcurB3nbfv373d6jeuad3Tt2tX8u2nTpo3j9sQTTzitw3XNe/Ta1qBBA1uZMmVsTZs2tdWrV8+2detWr1zXCMrpzJ8/3xYaGuoIAbt27bJFRkbapkyZ4v6njetKTU21/e1vfzPnWy9UVkE5ISHBVrp0advo0aPN8ytXrtjuuusuW6tWrXK0DnLmxIkTtpdfftmcW/tnNWzYMPNHSHJysmO9AQMGmCAdFxdnnk+dOtVWsGBBp1DgyjrIma+++sr2448/Op4fOnTIVrJkSdsLL7yQo+sa1768k5iYaH5J6x+cGYMy1zXvBuXHHnss23W4rnmH/t7R69iIESPM7xwVGxtr+/bbb71yXSMop9OjRw9bly5dnE7Q0KFDs2zhhOdlFZRnz55tK1CggO3MmTOOZcuWLTO/eHbu3OnyOsi9zZs3m3OqrWT2X/YaeN9//33HOnpx0zBtD2yurAPPaNGihe2RRx7J0XWNa1/eeeihh2wjR460rVixIlNQ5rrm3aCsrYv6h+bBgwdtaWlpTq9zXfOecePGmZbk9I0xGd3I6xqd+dLR+timTZs6laa0aNHCzAqoNWTw7mdTtWpVp/pL/Wzsr7m6DnLvxx9/lAIFCphzba/1u3z5stO/HX1dJwqyn3dX1oF7rl69aiZUWrZsmfzjH/8wkzGNGjUqR9c1rn15Y86cOWb2WJ0cywrXNe+aNWuWDB061PSJ0RrY9evXO17juuY9y5cvN7Mra02y/hvRieXS0tKc1rmR1zWCcjpa2J+xI4w+119E8fHxrn/KuCGfTdGiRSU0NNTRIcOVdZD7jhVPPfWUPPbYY1KsWDHHeVdW/3bSfzbXWwfuSUpKMgF5zJgxMnXqVBk4cKCZ0TQn1zWufXnzb0X/YJk9e7aEh4dbrsN1zXsGDBhgOsFu3bpVjh8/Li1btpSePXuajl/2z0ZxXbvxjh07JpcuXZL69evLQw89JK1atTKzMv/8889eua4RlNPRQKW9KzP+ElIFCxbMyeeMG/DZpKammpv9s3FlHeTu4tW5c2dp3bq1TJo0yemzUVb/dtJ/NtdbB+4pUqSIaVHWX/jaCqbB7O9//3uOrmtc+zzv4YcfNv9eEhISzOezbds2s3zz5s2mB35W553r2o3x4IMPOv7Y11FIpkyZYoLz//73P8dno7iu3Xh67hctWiSffPKJua4dOnRIateubT4zb1zXCMrp6LBXOsRSevo8KirKtEzCu5+NBrX0M67bn+uQWK6uA/foeezQoYPUqVNHvvzyS8cvEft5V1b/dtJ/NtdbB7mn5/mBBx6Qb7/9NkfXNa59nlemTBk5cOCAae3X27vvvmuWv/zyy7JgwQLHeee65hsiIyOlUKFCjn8rXNe8p2rVqtKsWTNzs4fawYMHy65du+TMmTM3/LpGUE5Ha2IWL15smuXt9IKmy+Fd+hnoP5D0NWT62eg4vG3atHF5HeScfi2pIblGjRoyb968TH+J16xZU6pVqyYLFy50LNM62S1btjj+7biyDnIuMTEx07LY2Finrxtdua5x7cub+mRtSbbftCxGffHFF/L44487zjvXtRtP+0toy316q1evNq2P+nW/4rrmPZ07dza/d9LXJevvC/3dY/8W4IZe13LU9S/AHTt2zPS07N27t23hwoW24cOHm7Evt23b5u1DyxdjJq5Zs8b24IMP2mrWrGke680+NIzq06ePrXr16rZPP/3U9vbbb9uKFClie+mll5z248o6cN2FCxdsN998s61GjRpmBBH756K38+fPO9bT8ZBDQkJsEydOtM2bN8/WrFkzW/PmzZ0+P1fWQc5oj24dJ/Trr7+2LViwwPTo1nO8ePHiHF3XuPblPatRLxTXtRvv8OHDtkaNGtnefPNN23fffWd74403zL+Re+65x2n0C65r3hEfH29+5/Tv398MCffOO++Y4eLGjh3rletakP4nd9k/sGjvSu2hrLOQ6VfC+pd/w4YNvX1YAU8L9n/77bdMy5csWWJqMO2tAFpHpr37taasT58+8qc//clpfVfWgeu0dXLQoEGWr7322mumA4ydft0/ffp0M+ueLtc6Wf06Mz1X1oHrtMPL22+/bVrDtPVF6/iGDx/u1JnP1esa1768pT3wtXPf3LlznWYH47rmvc6Wb731lpnFtWzZstKpUyfp379/pplcua55h9aL6zVLa5RLlSol9957r6lRTv/53KjrGkEZAAAAsECNMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAGAxNbROQXzx4kWfOzc6Mc9XX30l3333nVvHrRNc6Do66UtWdPISXefs2bO5WsfXBcJ7AJC3CMoAkMHp06elX79+cuLECZ86NzNmzJDmzZvLrFmzZMWKFW4dd3x8vFnnwIED2YZpXcdqtsycrOPrAuE9AMhbIXm8fwCAh7z33nsyduxYGTdunNv70Knd+/btK8WLF+dzAYDroEUZgM9JX0Kwb98+WbRokfz8889O65w/f96sc+XKFcey1NRUp6/S0+8nNjZWvv76a6f9/PLLL7JgwQLzWlb27Nljttu2bVuWLbRaBqG3kydPZvk+tm/fLvPmzZODBw9m+bO0JXjhwoVmXwkJCY7l+h51P/v375e9e/eaxzt37sz2HGZ13jQo9+zZUyIjI52Wa6uqngs9zqy4so6r5yOr48soJ9tkdf48+R7sn8eaNWvMsRw5ciTb4wfg52wA4GP2799v08tT9+7dbXXr1rV17drVVqRIEduoUaMc6/z4449mnfPnzzuWJSQkmGXr16932k+7du1sDRo0sN1999220NBQ2+jRo239+vWzNWzY0NalSxdbWFiY7cMPP8z083X9mjVr2jp16mQrXLiw7bHHHnM6zrlz59qKFy9uu/32281+IiMjbW+++Wam/XTr1s1Wp04dW58+fWxr1qyxfM9Tp061hYeH29q3b29r3LixrUSJErYVK1aY1xITE219+/Y156BZs2bm8YIFC9w6b6dPnzbrxMTEOJZNmjTJnIM777zTVr9+ffO+059HV9dx9Xxkd3zuvKfrnT9PvofDhw/bqlevbrbv0aOHeTx+/Pgsjx+AfyMoA/A59nA0ePBgW1pamlm2cuVKW1BQkO3gwYM5DspDhw517GfatGlm2ciRIx3b/fvf/7ZFR0dn+vl33XWX7fLly2bZxo0bbcHBwbbVq1eb5/v27bNFRESY47LTYypUqJBt586dTvt54IEHbFevXs3y/f72228mwM+ZM8exTINg1apVbcnJyY5lNWrUMMefm/OWMSjv2bPHVqBAAdvXX39tnut2+kdE+vPoyjo5OR/ZHZ8778mV8+ep9/Cvf/3L1rp1a8fr+rnOnz8/y88EgH+j9AKAzxo+fLgEBQWZx7fddpsEBwebUoicGjZsmGM/rVq1ciyz02WHDx+WpKQkp+3+9re/SWhoqHncokULad++vXz++efm+aeffiolS5Y0Hei++OILs1xLA0qUKGG+lk/vr3/9qzn2rHz55ZcSHR1taoftnnrqKdPhbuPGjXl63vRn16pVS7p162ae63ZPPvlkjtfJyflw53PNbhtXzp+n3kN4eLgp7Tl69Kh5rsdx7733ZnvsAPwXnfkA+CwNKHYFChSQkJAQSU5OzvF+0ndc0xrdrJalpKSYIGRXtWpVp/1Uq1bNUWOsIUxHTZg7d67TOm3btpVSpUo5LStfvny2x6f7rF69utOysmXLSkRERLY1zZ44b4cOHbJ8nzldJyfnw53PNbttXDl/nnoPI0aMkK1bt5rQXbduXenYsaOMHDlSKlasmO3xA/BPBGUAfsneQqtj4dq5E6Kzox0GMz63B6ZixYqZsK0dza7H3hKaFd1nxpZjDe2XLl3KFDI9TVtQf/rpp2zftyvr5OR8eJor589T76Fo0aIye/Zs08nwhx9+kKlTp0qzZs1Mh1AN5gACC6UXAPySvQUv/YgVVmML58b8+fMdj+Pi4mT58uXSpk0b87xLly5m5IlVq1Y5baOjLWQ14kJWtJRAR3LQES3stFWzUKFC0qRJk1y/j+v9bA2QWnpip6Nz5HQdT56PnHLl/HnqPdhLLjQUd+rUSd544w0z2oY7Lf8AfB8tygD8kn61rkFlyJAh8uijj8rx48fl448/9ujP0P3pkHM33XSTGcNY62AHDhxoXrvrrrtMnbPWvI4aNUpq1Kghu3fvNrPmff/996bl0VX6Pu655x7zNb7WRWsof/nll+WZZ54x7zMvde7cWW6//XbzfvR9aOibOXNmjtfx5PnIKVfOn6few7Rp02T9+vXm50VFRcknn3wit9xyi9SuXTvP3h8A76FFGYDP0dY67ZiVMVw98MADTrWg2iI4YMAA2bBhg+l0t3LlSrOd/et2q/3o1+u6rHDhwk71r7qsYMGCTtutXr1aKlSoIJs2bTIdtrRDl71zn3rnnXdMy6WO8btu3ToTyjRE2Wtfs3ofVrSzmXZA0/rXY8eOyWeffZZpYhENcDVr1szVebOacETHiX744Ydly5YtphVW30P68+jqOu6ej4yfa07fk6vnzxPvYcKECSaA6x9m+v9dr169zP8XWjMNIPAE6dAX3j4IAAAAwNfQogwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAgAWCMgAAAGCBoAwAAABYICgDAAAAFgjKAAAAgAWCMgAAACCZ/T/MuP7UaGgkngAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 800x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 6.2 Hyperparameter Tuning: Hidden Nodes Sweep\n",
    "hidden_nodes_options = [10, 50, 100, 200, 500]\n",
    "performances_hn = []\n",
    "optimal_lr = 0.2 # Fixed learning rate from previous experiment\n",
    "\n",
    "print(\"Starting Hidden Nodes sweep. This will take a while...\")\n",
    "\n",
    "for hn in hidden_nodes_options:\n",
    "    print(f\"Training network with {hn} hidden nodes...\")\n",
    "    testANN = ann(784, hn, 10)\n",
    "    \n",
    "    # Train 1 epoch\n",
    "    for record in list: \n",
    "        values = record.split(\",\")\n",
    "        data = np.asarray(values[1:], dtype=float) / 255.0 * 0.99 + 0.01\n",
    "        target = np.zeros(10) + 0.01\n",
    "        target[int(values[0])] = 0.99\n",
    "        testANN.backpropagation(data, target, optimal_lr)\n",
    "        \n",
    "    # Evaluate on the Test Set\n",
    "    score = 0\n",
    "    for record in list2:\n",
    "        values = record.split(\",\")\n",
    "        correct_label = int(values[0])\n",
    "        data = np.asarray(values[1:], dtype=float) / 255.0 * 0.99 + 0.01\n",
    "        \n",
    "        outputs = testANN.feedforward(data)\n",
    "        if np.argmax(outputs) == correct_label:\n",
    "            score += 1\n",
    "            \n",
    "    # Calculate performance\n",
    "    performance = score / len(list2)\n",
    "    performances_hn.append(performance)\n",
    "    print(f\"Performance for {hn} nodes: {performance:.4f}\\n\")\n",
    "\n",
    "# Plotting the exact graph requested\n",
    "plt.figure(figsize=(8, 5))\n",
    "# marker='D' creates the diamond shapes seen in the reference image\n",
    "plt.plot(hidden_nodes_options, performances_hn, marker='D', markersize=6, color='#003366', linewidth=1.5)\n",
    "\n",
    "# Formatting axes\n",
    "plt.xlabel(\"number of hidden nodes\")\n",
    "plt.ylabel(\"performance\")\n",
    "\n",
    "# Setting axes limits and ticks to match the image\n",
    "plt.xlim(0, 600)\n",
    "plt.xticks(np.arange(0, 601, 100))\n",
    "plt.ylim(0.6, 1.0)\n",
    "plt.yticks(np.arange(0.6, 1.05, 0.05))\n",
    "\n",
    "# Adding horizontal grid lines\n",
    "plt.grid(axis='y', linestyle='-', alpha=0.7)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1ce2d95",
   "metadata": {},
   "source": [
    "## 7. Model Selection and Architecture Evaluation\n",
    "\n",
    "**Which is the best ANN and how do you select which network is better?**\n",
    "The best network is the one that achieves the highest accuracy on the Test Set while keeping the Cost Function (Loss) to a minimum, without falling into Overfitting. It is selected by comparing different combinations of hyperparameters (Hidden nodes and Learning Rate), evaluating them with data the network never saw during training. The winning network is the one that best generalizes to new data, not the one that memorizes the training data.\n",
    "\n",
    "## 8. Microcontroller Deployment Strategy\n",
    "\n",
    "**How do we implement it on a microcontroller?**\n",
    "Once the network is trained on the computer, we extract the final weight matrices ($W_{ih}$ and $W_{ho}$) and export them as constant arrays (`const float`) in C/C++ language. On the microcontroller, only the inference stage (Feedforward) is programmed (matrix multiplication and the sigmoid function). The Backpropagation algorithm is completely omitted, which saves the microcontroller's limited memory and processing capacity, allowing for real-time signal processing."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.14.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}