1. Loaders are created by Sources for Servable Versions, then Loaders are sent as Aspired Versions to the Dynamic Manager, which loads and serves them to client requests.
  2. The Loader contains the metadata that needs to load the Servable.
  3. The Source uses a callback function to notify the manager of the Aspired Version.
  4. The manager applies the configured Version Policy to find the next action.
  5. If the manager specifies that it’s safe, it provides the required resources to the Loader and tells the Loader to load the new version.
  6. Clients request the manager for the Servable, either determining a version explicitly or just requesting the latest version. The manager returns a handle for the Servable. The Dynamic Manager applies the Version Policy and decides to load the new version.
  7. If there is enough memory the Dynamic Manager tells the Loader. The Loader instantiates the TensorFlow graph with the new weights.
  8. A client requests a handle to the latest version of the model, and the Dynamic Manager returns a handle to the new version of the Servable.
PreviousNext