Handwritten Digit Recognition with Neural Networks

The neural network architecture consists of a multi-layer feedforward design with two hidden layers. The input layer processes 28x28 pixel images (784 neurons total) from the MNIST dataset, which contains 60,000 training images of handwritten digits. The network structure flows from input through two hidden layers (40 and 20 neurons respectively) to an output layer with 10 neurons, each representing one digit class (0-9). The implementation uses the sigmoid activation function throughout all layers, which transforms the weighted inputs into values between 0 and 1. Training is performed using the backpropagation algorithm, which calculates gradients through the network layers in reverse order to update weights and biases. The mean squared error (MSE) cost function measures the difference between predicted and actual outputs during training. The training process runs for multiple epochs, processing each image in the dataset and adjusting network parameters using a learning rate of 0.07 (Found through trial and error, point of improvement). The model tracks accuracy after each epoch, providing feedback on the learning progress. Once trained, the network can classify handwritten digits with a high degree of accuracy. A key feature of this project is the interactive graphical interface built with Tkinter. Users can draw digits on a 280x280 pixel black canvas using their mouse, and the application automatically resizes the drawing to 28x28 pixels to match the MNIST input format. When the user clicks "Save Image", the neural network processes the drawing through forward propagation and displays both the input image and a bar chart showing the probability distribution across all 10 digit classes, highlighting the model's prediction. This project was of great educational value for me, since by implementing neural network fundamentals from scratch I was able to understand how neural networks work at a low level. All matrix operations, gradient calculations, and weight updates are performed manually using NumPy, providing transparency into the mathematical operations that power modern machine learning systems. Limitations: While this project is effective as a learning tool, and a very fun project to build, it has several limitations. The network architecture and training routine are intentionally simple and not optimized for maximum performance or speed, which means training can be relatively slow and accuracy may not match modern deep learning models built with specialized libraries like TensorFlow or PyTorch. The model is trained and evaluated only on the MNIST dataset, so it does not generalize to more complex or noisy real-world digit data. Additionally, the use of the sigmoid activation function and mean squared error loss is pedagogical rather than state-of-the-art, and there is no use of techniques such as regularization, advanced optimizers, or batching. Possible Next Steps: Potential next steps for this project include dynamic learning rate tuning, experimenting with different activation functions (such as ReLU) and loss functions (like cross-entropy) to improve training dynamics and accuracy. The architecture could be extended with additional layers or converted into a convolutional neural network (CNN) to better capture spatial patterns in image data. From an engineering perspective, batching, learning rate schedules, and more advanced optimization algorithms (e.g., Adam, RMSprop) could be added. As discussed before, I'm currently learning about Machine Learning at university, this project was made when all I knew was the very basics of Machine Learning, so I'm looking forward to implementing these techniques in this project and see how much I can improve it.