I have trained a classification model in sklearn. Now I want to save this model in order to use it inside my Flask web application. I have found that this can be done using Joblib package. But the file with the model becomes extremely large, and I am afraid about how will it work in the web application. Are there any other methods of saving the machine learning models?
What is the problem with having a large file with the model inside web application? Do you have limited disk space on the webserver?
Actually I have not deployed my web app yet, so I just asking about is it ok or not. I know that in web development everybody tries to minimize the size of data, files, etc.
This is true if you need to pass data over the network. For example, if you need to send it from back-end to front-end, or to some 3rd party API. But if the model will lay statically on the server, it doesn't matter.
However, in order to save disk space, you can think about how to minimize the size of the file. Try using the pickle library, for example. This is a standard way for serializing Python objects.
As far as I know, joblib is better for saving trained sklearn models (but I am not sure). Did you try to use compression? Joblib allows models compression. You can even set the extent of compression. But the more compressed files can require more time to load them before inference, and this can become a problem for the web application, where the speed is what really important.
Thanks, everybody. I have tried joblib with compression=3, and this works very well! The speed of loading is decent, and the size of the file becomes significantly smaller.
Just drop us an email to ...
Show more 
Log in with Google