Reverse engineering (AML part 2)

To better understand how the neural network from the book works, reverse engineering is needed. For example, understand how an image library converts an image into a digital set and sends them to the input layer of a neural network?

I’m using PyCharm code editor, so the code for IPython won’t work, so it will be slightly different. To create a neural network that will work in an automatically learning program, need to break the program into three stages. The first stage is creating a neural network and generating weights. The second stage is training the neural network and storing the weights for their subsequent use. The third stage is to write a program that will apply ready-made weights for object recognition.

To successfully complete these three points, necessary to understand how the code from the book of Tariq Rashid works. This understanding also helps to understand another, more complex book by Simon Haykin, “Neural Networks”, second edition. Having at your disposal these two powerful sources of knowledge, realy not only create neural networks and train them, but also optimize all processes in the neural network at the mathematical level. For me, Tariq Rashid’s book was the beginning of the path to more complex and exciting mathematics.

Link to Tariq Rashid’s blog https://makeyourownneuralnetwork.blogspot.com/

I’m not promoting this author, it’s just that his book really helped me)

Moving on, studying the program code and mathematical principles, I broaden my horizons. However, for full-fledged artificial intelligence, neural networks are clearly not enough, or rather, this is only a small fraction of what should be.

So, in order to save the weight coefficients of a neural network, necessary to understand where the weight coefficients already adjusted after training come from. They are in the query function. Necessary the arrays obtained after applying the sigmoid. Since there are only two weight matrices in this example, there will only be two arrays to store. For stored a these datasets maybe use various variant. For example, text document or special databases such as graph databases (SPARQL), or file with the h5py extension.

Referring to the source code, the link to the source code is https://github.com/makeyourownneuralnetwork/makeyourownneuralnetwork/blob/master/part2_neural_network.ipynb. For the convenience of its use in PyCharm I will rewrite it like this:

The query function returns already prepared weights, so they only need to be saved. The next step is to create a neural network class that will use the stored weights.

To better understand how the code works, I needed to write and test several of my own versions of the code. First I tried to write matrix multiplication without using libraries like nampy or pandas.

The result of the program’s work shows a new matrix. Here I was interested in the multiplication process itself, so I did not write a generator or add a sigmoid.

To better understand how a simple neural network, consisting of three layers and two neurons in each layer, works, I wrote a simple program for myself. The program is written in a procedural style to make it as easy as possible to understand the processes occurring in the neural network.

It was interesting to study the work of the imageio. I printed out the work of the program gradually, I watched how it works line by line and what it does. Above the original code, below mine. The same will need to be done with pyaudio if I write speech recognition or text-to-speech (TTS / STT).

Thus, exploring the program step by step, I found a way to save the weights, figured out how to create my own versions of a neural network, how to connect libraries to work with input data. In the course of the study, I had many questions, one of them sounds like this: How to automate the process of building a neural network? How to determine the required number of layers, neurons in a layer and the number of learning epochs? I already heard about such developments, but I wanted to do something of my own. Maybe I’ll get lucky and my solution will be better?

The next step is to write a program that uses ready-made weighting factors in the image recognition program.