Thinking out loud (AML part 3)

It is not difficult to maintain the weights. However, there are many factors to consider. For example, maybe need information about which neural network the weights belong to. To do this, necessary to record metadata of the neural network, such as the number of layers, the number of neurons in each layer, what exactly does the neural network do, what is the activation function of the network?

It would be even more convenient to assemble a neural network as a constructor or even to reduce everything to a simple graphical interface in which the user will enter only the network parameters, and the program will take care of everything. Well, that’s a pretty daring idea.

To accomplish this task, necessary to break the program into the required components. For example, when specifying the number of neurons in each layer, can simply pass this data into variables. But when the number of layers is more than three, will have to extend the initialization function, which is not very convenient. The training function will also need to be improved.

Where to begin?

The following strategy can be applied here. To begin with, it is enough to take out of the brackets all the variables that do not require additional processing in the program. These are variables that are entered by the user. These are the number of neurons in each layer, the learning rate and the activation function. The number of layers in the network is more difficult, since weights need to be generated for every two adjacent layers. That is, need to generate the required number of matrices and then perform the necessary actions on them.

It will be more difficult with the learning function. Since the network is trained by the back propagation method, it is necessary to calculate the error for each layer. This difficult but achievable task.

The question may arise what is all this for? The fact is that for each task it is best to develop your own neural network to obtain the best result. To find out which option works faster and better, need to do tests, but first of all, need to find the best variants. Can go much further, for example, automate the search process. In this case, need to describe the selection criteria in the program itself. Will it be a kind of classifier? Wait, but isn’t a neural network a classifier? Here I come to the idea that to find the optimal solution in automatic mode, will need to again write a neural network.

What if go even further? On the Internet, I have come across articles that describe the use of neural networks not only as classifiers, but also as generators of new data variants. Is it possible to somehow use this for own purposes?

All of these are, of course, interesting ideas, but I forget about one more thing – hardware resources. Training a neural network is a tricky task, even for a good desktop computer, and this is much more resource-hungry code. What to do? I was lucky to live in a time when there are multi-threaded processors, as well as programming languages ​​capable of managing these threads. This is certainly not a quantum computer, but it is quite enough for my task. Such a program can be easily scaled in the future by uploading it to a remote server that can be rented.

Yes, the idea turned out to be very ambitious for me. I am will not to try, l will do it.