Source: model_server/core.py#L0

Servable

Helper class that provides a standard way to create an ABC using inheritance.

Servable.__init__

__init__(self)

Abstract base class for custom servables. All custom servables must inherit from this. All custom servables inheriting from this must implement the following methods:

predict(self, input_array_dict)
get_model_info(self, list_of_model_info_dict)

Servable.get_model_info

get_model_info(self, list_of_model_info_dict)

Abstract method which is responsible for the call GetModelInfo

Arguments:

  • list_of_model_info_dict (list/tuple): A list containing model_info_dicts

Note:

model_info_dict contains the following keys:

{
  "name": "model name as string"
  "version": "version as string"
  "status": "status string"
  "misc": "string with miscellaneous info"
}

Returns:

  • list_of_model_info_dict (dict): containing the model and server info. This is similar to the function input

Servable.predict

predict(self, input_array_dict)

Abstract method where the model prediction logic lives. This method is responsible for the gRPC call GetPredictions(). All custom servables must define this method.

Arguments:

  • input_array_dict (dict): The PredictionRequest proto decoded as a python dictionary.
# example
input_array_dict = {
  "input_tensor_name1": numpy array,
  "input_tensor_name2": numpy array
  }

Returns:

A python dictionary with key (typically output name) and value as numpy array of predictions

# example
output = {
  "output_tensor_name1": numpy array,
  "output_tensor_name2": numpy array
  }

ModelServerServicer

ModelServerServicer.__init__

__init__(self, custom_servable_object)

gRPC Model Server Services. This is where the RPC methods are defined.

Arguments:

  • custom_servable_object: custom servable classe's instance

ModelServerServicer.GetModelInfo

GetModelInfo(self, request, context)

Entrypoint for GetModelInfo gRPC call. Uses the get_model_info method defined in custom servable

Arguments:

  • request (protobuf): gRPC request containing input ModelInfo protobuf
  • context (protobuf): gRPC context object

Returns:

ModelInfo protobuf

ModelServerServicer.GetPredictions

GetPredictions(self, request, context)

Entrypoint for GetPredictions gRPC call. Uses the predict method defined in custom servable

Arguments:

  • request (protobuf): gRPC request containing input PredictRequest protobuf
  • context (protobuf): gRPC context object

Returns:

PredictResponse protobuf