Skip to content

model_server.utils.prediction_utils

create_predict_request

[view source] 

1
def create_predict_request(input_tensorproto_dict, name=None, version=None)

Creates a PredictRequest proto

Arguments:

  • input_tensorproto_dict dict - A dictionary with key (typically input tensor name string) and value as TensorProto.
  • name str - Optional model name. Default None
  • version str - Optional model version. Default None

Returns:

PredictRequest proto

create_predict_response

[view source] 

1
def create_predict_response(output_tensorproto_dict, name=None, version=None)

Creates a PredictResponse proto

Arguments:

  • input_tensorproto_dict dict - A dictionary with key (typically output tensor name string) and value as TensorProto.
  • name str - Optional model name. Default None
  • version str - Optional model version. Default None

Returns:

PredictResponse proto

model_server.utils.tensor_utils

create_tensor_proto

[view source] 

1
def create_tensor_proto(array, shape=None, dtype=None, name=None)

Create a TensorProto from numpy array.

Arguments:

  • array np.ndarray - The numpy array to convert to TensorProto
  • shape tuple - Optional shape of the array to reshape it to. If not given, it is inferred from the numpy array. Default None
  • dtype str - Optional dtype to convert the array to. Refer tensor.proto for supported datatypes. If not given, it is inferred from the numpy array. Default None.
  • name str - Optional name for the TensorProto. Default None

Returns:

A TensorProto containing the given array

Notes:

  • python strings will be encoded to python bytes.
  • Use dtype = "object" if the numpy array contains strings.
  • dtype "string" and "object" are treated as same.
  • "string" is converted to python "object". This is because, numpy handles variable length strings in this way

create_array_from_proto

[view source] 

1
def create_array_from_proto(tensor_proto)

Retrive array from TensorProto.

Arguments:

  • tensor_proto TensorProto - An instance of TensorProto

Returns:

numpy array

Notes:

  • python strings will be encoded to python bytes.
  • Use dtype = "object" if the numpy array contains strings.
  • dtype "string" and "object" are treated as same.
  • "string" is converted to python "object". This is because, numpy handles variable length strings in this way

model_server.utils.model_info_utils

create_model_info_proto

[view source] 

1
def create_model_info_proto(list_of_model_info_dict)

Creates a ModelInfo proto

Arguments:

  • list_of_model_info_dict list/tuple - A list containing model_info_dicts

Notes:

model_info_dict contains the following keys:

1
2
3
4
5
6
7
8
```python
{
    "name": "model name as string"
    "version": "version as string"
    "status": "status string"
    "misc": "string with miscellaneous info"
}
```

Returns:

ModelInfo proto

decode_model_info_proto

[view source] 

1
def decode_model_info_proto(model_info_proto)

Decodes the model_info_proto created by create_model_info_proto

Arguments:

  • model_info_proto ModelInfo proto - model_info_proto created by create_model_info_proto

Returns:

  • list_of_model_info_dict list - A list containing model_info_dicts

Notes:

model_info_dict contains the following keys:

1
2
3
4
5
6
7
8
```python
{
    "name": "model name as string"
    "version": "version as string"
    "status": "status string"
    "misc": "string with miscellaneous info"
}
```