# Development of a Modular Python Library from Scratch for Automated ROI Segmentation in Thermal Images

# Module 3: Artificial Neural Network (ANN)

Author: Sofia Samaniego Lopez

Institution: Universidad Autonoma de Baja California (UABC)

Advisor: Dr. Gerardo Marx Chavez Campos

This notebook presents **Module 3** of the library's development: the implementation of an **Artificial Neural Network (ANN) from scratch**.

With the objective of maintaining algorithmic transparency and bypassing commercial "black-box" frameworks, the entire network architecture (weight matrix initialization, feedforward propagation, and backpropagation via gradient descent) has been programmed using strictly linear algebra through **NumPy**. 

As a proof of concept and baseline evaluation, the model is trained and validated using the **MNIST** dataset. This demonstrates the pure mathematical algorithm's capability to classify complex patterns prior to scaling the framework for thermal image processing.

## 1. Environment Setup & Initialization
Importing core libraries for matrix operations and data visualization. A random seed is set to ensure reproducible weight initialization across experimental runs.


```python
!pip3 install numpy
!pip3 install matplotlib
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(12)
```

    Requirement already satisfied: numpy in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (2.5.0)
    Requirement already satisfied: matplotlib in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (3.11.0)
    Requirement already satisfied: contourpy>=1.0.1 in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (from matplotlib) (1.3.3)
    Requirement already satisfied: cycler>=0.10 in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (from matplotlib) (0.12.1)
    Requirement already satisfied: fonttools>=4.22.0 in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (from matplotlib) (4.63.0)
    Requirement already satisfied: kiwisolver>=1.3.1 in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (from matplotlib) (1.5.0)
    Requirement already satisfied: numpy>=1.25 in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (from matplotlib) (2.5.0)
    Requirement already satisfied: packaging>=20.0 in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (from matplotlib) (26.2)
    Requirement already satisfied: pillow>=9 in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (from matplotlib) (12.2.0)
    Requirement already satisfied: pyparsing>=3 in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (from matplotlib) (3.3.2)
    Requirement already satisfied: python-dateutil>=2.7 in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (from matplotlib) (2.9.0.post0)
    Requirement already satisfied: six>=1.5 in c:\Users\sofia\ANN-From-Scratch\.venv\Lib\site-packages (from python-dateutil>=2.7->matplotlib) (1.17.0)
    

## 2. Artificial Neural Network (ANN) Architecture

Neural Network's Basic Structure 


```python
class ann:
    #init
    def __init__():
        pass

    #feedfoward
    def feedforward():
        pass

    #backpropagation
    def backpropagation():
        pass

```


```python
MyANN = ann
print(type(MyANN))
```

    <class 'type'>
    

### 2.1 Initialization
Defining the network structure (input, hidden, and output layers). Synaptic weight matrices ($W_{ih}$ and $W_{ho}$) are initialized using a normal distribution to break mathematical symmetry.


```python
class ann:
    #init
    def __init__(self, inputNodes: int, hiddenNodes: int, outputNodes: int):
        # Nodes
        inN = inputNodes # Private var or parameters
        hN = hiddenNodes
        oN = outputNodes
        # Weights
        np.random.seed(12) #seed for reproducibility
        self.wih = np.random.randn(hN, inN) #weights for input to hidden layer
        self.who = np.random.randn(oN, hN) #weights for hidden to output layer
        pass

    #feedfoward
    def feedforward():
        pass

    #backpropagation
    def backpropagation():
        pass
```


```python
MyANN = ann(3, 3, 3)
```


```python
MyANN.wih
```




    array([[ 0.47298583, -0.68142588,  0.2424395 ],
           [-1.70073563,  0.75314283, -1.53472134],
           [ 0.00512708, -0.12022767, -0.80698188]])




```python
MyANN.who
```




    array([[ 2.87181939, -0.59782292,  0.47245699],
           [ 1.09595612, -1.2151688 ,  1.34235637],
           [-0.12214979,  1.01251548, -0.91386915]])



### 2.2 Feedforward (Inference)
So the next step is to create the network of nodes and links. The most important part of the network is the link weights. They’re used to calculate the signal being fed forward, the error as it’s propagated backwards, and it is the link weights themselves that are refined in an attempt to to improve the network.

For the basic NN, the weight matrix consist of:

- A matrix that links the input and hidden layers, $Wih$, of size hidden nodes by input nodes ($hn×in$)
- and another matrix for the links between the hidden and output layers, $Who$, of size $on×hn$ (output nodes by hidden nodes)

$$X_h=W_{ih}I$$
$$O_h=\sigma(X_h)$$


Then, 

$$X_o=W_{ho}O_{h}$$
$$O_o=\sigma(X_o)$$


```python
class ann:
    #init
    def __init__(self, inputNodes: int, hiddenNodes: int, outputNodes: int):
        # Nodes
        inN = inputNodes # Private var or parameters
        hN = hiddenNodes
        oN = outputNodes
        # Weights
        np.random.seed(12) #seed for reproducibility
        self.wih = np.random.randn(hN, inN) #weights for input to hidden layer
        self.who = np.random.randn(oN, hN) #weights for hidden to output layer
        pass

    #feedfoward
    def feedforward(self, Inputs):
        # Forward pass to hidden layer
        inputs = np.array(Inputs, ndmin=2).T
        Xh = np.dot(self.wih, inputs)
        af = lambda x: 1 / (1 + np.exp(-x))
        Oh = af(Xh)
        # Forward pass to output layer
        Xo = self.who @ Oh
        Oo = af(Xo)
        return Oo

    #backpropagation
    def backpropagation():
        pass
```


```python
MyANN = ann(3, 3, 3)
MyANN.feedforward([0.1, 0.2, 0.3])
```




    array([[0.80230104],
           [0.65960645],
           [0.48247944]])



### 2.3 Backpropagation (Training)
The core learning algorithm. It calculates the prediction error, propagates it backward, and dynamically updates the weight matrices using gradient descent and the chain rule.


The Gradient of Error
$$ \frac{\partial E}{\partial w_{ho}}= -e_o\cdot \sigma \left(w_{ho} O_h\right) \left(1-\sigma\left (w_{ho} O_h\right) \right) O_h $$

Thus,

$$ \frac{\partial E}{\partial w_{ho}}= -e_o\cdot O_o \left(1-O_o \right) O_h $$


```python
class ann:
    #init
    def __init__(self, inputNodes: int, hiddenNodes: int, outputNodes: int):
        # Nodes
        inN = inputNodes # Private var or parameters
        hN = hiddenNodes
        oN = outputNodes
        # Weights
        np.random.seed(12) #seed for reproducibility
        self.wih = np.random.randn(hN, inN) #weights for input to hidden layer
        self.who = np.random.randn(oN, hN) #weights for hidden to output layer
        pass

    #feedfoward
    def feedforward(self, Inputs):
        # Oh
        inputs = np.array(Inputs, ndmin=2).T
        Xh = np.dot(self.wih, inputs)
        af = lambda x: 1 / (1 + np.exp(-x))
        Oh = af(Xh)
        # Oo
        Xo = self.who @ Oh
        Oo = af(Xo)
        return Oo

    #backpropagation
    def backpropagation(self, Inputs, Targets, Learning):
        lr = Learning
        inputs = np.array(Inputs, ndmin=2).T
        targets = np.array(Targets, ndmin=2).T
        
        # 1. Internal feedforward
        Xh = self.wih @ inputs
        af = lambda x: 1 / (1 + np.exp(-x))
        Oh = af(Xh)
        
        Xo = self.who @ Oh
        Oo = af(Xo)
        
        # 2. Error calculation
        Eo = targets - Oo
        Eh = self.who.T @ Eo

        # 3. Weight matrices update
        self.who = self.who + (lr * Eo * Oo * (1-Oo) ) @ Oh.T
        self.wih = self.wih + (lr * Eh * Oh * (1-Oh) ) @ inputs.T
        pass
```


```python
MyANN = ann(3, 5, 3)
MyANN.backpropagation([0.1, 0.2, 0.3], [0.01, 0.01, 0.99], 0.3)
```

## 3. MNIST Dataset Exploration
Loading the training dataset. To verify the geometric structure, a raw 784-pixel flat array is extracted and reshaped into a 28x28 2D matrix for visual confirmation.


```python
# Load training data
file = open("mnist_train.csv")
list = file.readlines()
file.close
```




    <function TextIOWrapper.close()>




```python
# Visualize sample at index 120
values = list[120].split(",")
image = np.asarray(values[1:], dtype=int)
plt.imshow(image.reshape(28,28), cmap='Grays')
plt.show()
```


    
![png](README_files/README_24_0.png)
    



```python
values [0]
len(list)
```




    49999



## 4. Model Training
Setting up hyperparameters. During training, pixel intensities are normalized to a $[0.01, 1.0]$ range to prevent zero-gradient issues. Target labels are formatted using an adapted One-Hot Encoding.


```python
# hyperparameters
inputNodes = 784
hiddenNodes = 100
outNodes = 10
learningRate = 0.1
MyANN = ann(inputNodes, hiddenNodes, outNodes)
```


```python
# Iterative training loop
epoch = 1
for e in range (epoch):
    for record in list: 
        values = record.split(",")
       
        # Input data normalization
        data = np.asarray(values[1:], dtype=int)/255*0.99+0.01
        index = np.asarray(values[0],dtype=int)
       
        # Target Vector construction
        target = np.zeros(outNodes) + 0.01
        target[index] = 0.99
        
        MyANN.backpropagation(data, target, learningRate)
        pass
    pass
        
```

## 5. Validation & Inference
Evaluating model performance using unseen test data. A new sample is normalized and processed to extract the final prediction vector, which is then visually compared to the ground truth image.


```python
# Load testing data
file2 = open("mnist_test.csv")
list2 = file2.readlines()
file2.close
```




    <function TextIOWrapper.close()>




```python
# Inference on sample 500
values = list2[500].split(",")
data = np.asarray(values[1:], dtype=int)/255*0.99+0.01

# Display probability vector for the 10 classes
MyANN.feedforward(data)
```




    array([[1.12958417e-03],
           [7.04806122e-03],
           [1.33450332e-03],
           [9.98132913e-01],
           [1.21412650e-04],
           [6.66539345e-03],
           [2.96287176e-05],
           [1.83308214e-03],
           [8.11709452e-04],
           [1.75208890e-04]])




```python
# Visual verification
image = np.asarray(values[1:], dtype=int)
plt.imshow(image.reshape(28,28), cmap='Grays')
plt.show()
```


    
![png](README_files/README_32_0.png)
    

