Exist huge number of database management systems (DBMS). Of course, there are graph databases as well. However, for ease of understanding, I settled on MySQL. There are two reasons for this. The first is that I have experience with this DBMS. Secondly, this is a large amount of previously written program code for working with MySQL that I can use in this project.
For storing the neural network, I chose MySQL. Hence, I will be using mysql.connector. To work with this library, I wrote a class in which I laid the basic functions for quick integration into the program.
From the last part, it became clear what needs to be saved. I am creating a table in which I will save all the parameters of the neural network.
I have a query function from which I can get the weights. Necessary to modify function so that the weights are saved to the database. To determine the variables in which the required data is located, I’ll do a simple check.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
def query(self, inputs_list): inputs = numpy.array(inputs_list, ndmin=2).T hidden_inputs = numpy.dot(self.wih, inputs) hidden_outputs = self.activation_function(hidden_inputs) print(hidden_outputs) final_inputs = numpy.dot(self.who, hidden_outputs) final_outputs = self.activation_function(final_inputs) print(final_outputs) return final_outputs |
In this case, these are the self.who and self.wih variables. This can be checked by implementing the function in the program
|
1 2 |
final_inputs = numpy.dot(self.who, hidden_outputs) print(self.who) |
In this case, the output weights after training are the same as those in the self.who variable. These arrays (lists) will be identical. Therefore, the variable self.wih contains the weights between the first and second layers. It is worth noting that as the number of layers increases, the number of columns in tables will also increase. This means that will need to automate this process. There is such a possibility in SQL language.
Now need to think about the presentation of the data in the database. For MySQL, do not need to cast the float type to the string type, as necessary do when saving to a text document. It should be borne in mind that a table in the database should be created automatically for each new version of the neural network. Also need to understand that the number of hidden layers in a neural network can change as well as the number of neurons in each layer. The number of layers and neurons can be set by both the user and the algorithm for automatic optimization. Thus, it may seem that the task is impossible. However, if you move towards the goal gradually, step by step, then everything will become clearer. I think that gradually I will be able to reduce everything to the fact that another neural network will perform the search for the best variant of the neural network. Already at this stage, the hierarchy of algorithms in the program is visible.
I am using the OpenServer Windows program which has phpmyadmin. I created a database on the local server “neural_networks” and added the “weights” table.
|
1 2 3 4 5 6 7 |
CREATE TABLE `weights` ( `id` int(11) NOT NULL, `inputs` int(11) NOT NULL, `outputs` int(11) NOT NULL, `lr` int(11) NOT NULL, `hidden_1` float NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci; |
Perhaps need to add additional fields, which will describe the neural network, what it does and so on. But in my opinion this is enough at this stage.
Oops! Houston, we have a problem … MySQL is not good for keeping weights!
On the one hand, we need a way to save neural network data independently of remote servers. This means you cannot use Keras or TensorFlow. What to do?
The fact is that neural networks are saved in special hierarchical data formats, in the hdf5 format. But MySQL doesn’t allow this. MySQL cannot build a hierarchy of tables or databases. However, you can define a similar structure in the program code. Ultimately, the goal is to save the data.
You can achieve the same result with CSV files. In this case, each file will contain a separate table. This option does not require the installation of a MySQL server, but it creates a load on the physical media when reading and writing data. However, CSV does not require a MySQL server.