Global AI and Data Science

Global AI & Data Science

Train, tune and distribute models with generative AI and machine learning capabilities

 View Only
Back to Library

Easier Model Serving with zerocopy 

Fri August 19, 2022 04:07 PM

A comparison of the code to deploy our example intent detection model, before and after applying zero-copy model loading to the deployment. Only two lines of code need to change. See below for a detailed description of the changes.

In a previous post, we introduced the concept of zero-copy model loading. Zero-copy model loading involves keeping the weights of a machine learning model in shared memory so that different processes can load the model for inference instantly without copying data.



#Featured-area-1
#GlobalAIandDataScience
#GlobalDataScience
#Highlights-home
#In-the-know-feature

Easier Model Serving with zerocopy
Statistics
0 Favorited
8644 Views
0 Files
0 Shares
0 Downloads