Skip to content

model_server.core

Servable Objects

[view source] 

1
class Servable(abc.ABC)

__init__

[view source] 

1
def __init__()

Abstract base class for custom servables. All custom servables must inherit from this. All custom servables inheriting from this must implement the following methods:

1
2
predict(self, input_array_dict)
get_model_info(self, list_of_model_info_dict)

predict

[view source] 

1
2
@abc.abstractmethod
def predict(input_array_dict)

Abstract method where the model prediction logic lives. This method is responsible for the gRPC call GetPredictions(). All custom servables must define this method.

Arguments:

  • input_array_dict dict - The PredictionRequest proto decoded as a python dictionary.
1
2
3
4
5
# example
input_array_dict = {
                   "input_tensor_name1": numpy array,
                   "input_tensor_name2": numpy array
                    }

Returns:

A python dictionary with key (typically output name) and value as numpy array of predictions

1
2
3
4
5
# example
output = {
           "output_tensor_name1": numpy array,
           "output_tensor_name2": numpy array
          }

get_model_info

[view source] 

1
2
@abc.abstractmethod
def get_model_info(list_of_model_info_dict)

Abstract method which is responsible for the call GetModelInfo

Arguments:

  • list_of_model_info_dict list/tuple - A list containing model_info_dicts

Notes:

model_info_dict contains the following keys:

1
2
3
4
5
6
7
8
```python
{
    "name": "model name as string"
    "version": "version as string"
    "status": "status string"
    "misc": "string with miscellaneous info"
}
```

Returns:

  • list_of_model_info_dict dict - containing the model and server info. This is similar to the function input

ModelServerServicer Objects

[view source] 

1
class ModelServerServicer(server_pb2_grpc.ModelServerServicer)

__init__

[view source] 

1
def __init__(custom_servable_object)

gRPC Model Server Services. This is where the RPC methods are defined.

Arguments:

  • custom_servable_object - custom servable classe's instance

GetPredictions

[view source] 

1
def GetPredictions(request, context)

Entrypoint for GetPredictions gRPC call. Uses the predict method defined in custom servable

Arguments:

  • request protobuf - gRPC request containing input PredictRequest protobuf
  • context protobuf - gRPC context object

Returns:

PredictResponse protobuf

GetModelInfo

[view source] 

1
def GetModelInfo(request, context)

Entrypoint for GetModelInfo gRPC call. Uses the get_model_info method defined in custom servable

Arguments:

  • request protobuf - gRPC request containing input ModelInfo protobuf
  • context protobuf - gRPC context object

Returns:

ModelInfo protobuf