Saving neural network weights to csv file (AML part 6)

Saving the neural network weights to a csv file is much easier. In my opinion this option is faster and csv tables are easily exported to other databases. Another reason is the great versatility of this approach, since to save to the database, you need to raise the local server, while for the files you just need to allocate a directory.

Thus, it is more profitable to store neural network weights in csv files. I decided to create a separate file for each table. Perhaps this is not a very good solution, since with a large number of layers it can get confused. However, it works well for now.

To save to csv, I needed to understand how the built-in csv library in python works. I wrote the first test code, this is what I got.

For a test recording, I created a data array. In its structure, it does not differ from the weights that are contained in the numpy array during generation. This is the same two-dimensional array. The write_csv function writes data to the dict_input.csv file, in this case the file can be named anything, which means that its name can be easily generated depending on the number of layers included in the neural network. The result is a table corresponding to those created in MySQL.

In the next step, I embedded the resulting csv_write function into the NN_save_csv.py file. I also added the float_type function to this version of the program. For some reason numpy does not communicate well with databases and files. After training the neural network, the weights were successfully saved to the files weights_1.csv and weights_2.csv.

The final step was reading the weights from the database and csv files. It was pretty straightforward. Summing up the work done, it is worth noting that both options can be useful. For example, the option using a database can be used in web services, while the option with csv will work fine in stand-alone systems without the ability to connect to the Internet.

the resulting programs using the stored weights are available from the links:

MySQL https://github.com/scisoftdev/Python/blob/master/Save_weights_of_NN/NN_read_mysql.py

CSV https://github.com/scisoftdev/Python/blob/master/Save_weights_of_NN/NN_save_csv.py

They turned out to be very simple, it will not be difficult to understand their work, and you can always ask your questions in the comments.

In the next article, I will show how the neural network generator works. I will show you how to save the metadata of a neural network and how to fully load a trained neural network. Perhaps, for such tasks, it is easiest to use popular libraries such as TensorFlow, Kafka, PyTorch, OpenCV. I partially agree with this opinion.

All comments containing advertising and spam will not be moderated and will be deleted!

Leave a Reply

Your email address will not be published. Required fields are marked *