如何在Tensorflow服務中進行批處理？

Question

部署的Tensorflow爲Inception-V3服務並運行測試。工作正常。

現在，想爲Inception-V3服務的批處理。 例如想要發送10個圖像進行預測而不是一個。

如何做到這一點？要更新哪些文件（inception_saved_model.py或inception_client.py）？這些更新是什麼樣子的？以及圖像如何傳遞給服務 - 它是以包含圖像的文件夾的形式傳遞的？

感謝您對此問題的深入瞭解。任何與此相關的代碼片段都會非常有用。

=================================

更新inception_client.py

# Copyright 2016 Google Inc. All Rights Reserved. 
# 
# Licensed under the Apache License, Version 2.0 (the "License"); 
# you may not use this file except in compliance with the License. 
# You may obtain a copy of the License at 
# 
#  http://www.apache.org/licenses/LICENSE-2.0 
# 
# Unless required by applicable law or agreed to in writing, software 
# distributed under the License is distributed on an "AS IS" BASIS, 
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 
# See the License for the specific language governing permissions and 
# limitations under the License. 
# ============================================================================== 

#!/usr/bin/env python2.7 

"""Send JPEG image to tensorflow_model_server loaded with inception model. 
""" 

from __future__ import print_function 

"""Send JPEG image to tensorflow_model_server loaded with inception model. 
""" 

from __future__ import print_function 

# This is a placeholder for a Google-internal import. 

from grpc.beta import implementations 
import tensorflow as tf 
from tensorflow.python.platform import flags 
from tensorflow_serving.apis import predict_pb2 
from tensorflow_serving.apis import prediction_service_pb2 


tf.app.flags.DEFINE_string('server', 'localhost:9000', 
          'PredictionService host:port') 
tf.app.flags.DEFINE_string('image', '', 'path to image in JPEG format') 
FLAGS = tf.app.flags.FLAGS 


def main(_): 
    host, port = FLAGS.server.split(':') 
    channel = implementations.insecure_channel(host, int(port)) 
    stub = prediction_service_pb2.beta_create_PredictionService_stub(channel) 
    # Send request 
    #with open(FLAGS.image, 'rb') as f: 
    # See prediction_service.proto for gRPC request/response details. 
    #data = f.read() 
    #request = predict_pb2.PredictRequest() 
    #request.model_spec.name = 'inception' 
    #request.model_spec.signature_name = 'predict_images' 


# request.inputs['images'].CopyFrom(
#  tf.contrib.util.make_tensor_proto(data, shape=[1])) 
# result = stub.Predict(request, 10.0) # 10 secs timeout 
# print(result) 


# Build a batch of images 

    request = predict_pb2.PredictRequest() 
 request.model_spec.name = 'inception' 
request.model_spec.signature_name = 'predict_images' 
   
  image_data = [] 
  for image in FLAGS.image.split(','): 
   with open(image, 'rb') as f: 
     image_data.append(f.read()) 
   
  request.inputs['images'].CopyFrom(
      tf.contrib.util.make_tensor_proto(image_data, shape=[len(image_data)])) 
   
  result = stub.Predict(request, 10.0)  # 10 secs timeout 
  print(result) 
if __name__ == '__main__': 
    tf.app.run() 

Answer 1

您應該能夠計算對inception_client.py中的請求構造代碼進行較小更改的一批圖像的預測。在該文件中以下行創建具有「間歇」含有單一圖像的請求（注意shape=[1]，這意味着「長度爲1的矢量」）：

with open(FLAGS.image, 'rb') as f: 
    # See prediction_service.proto for gRPC request/response details. 
    data = f.read() 
    request = predict_pb2.PredictRequest() 
    request.model_spec.name = 'inception' 
    request.model_spec.signature_name = 'predict_images' 
    request.inputs['images'].CopyFrom(
     tf.contrib.util.make_tensor_proto(data, shape=[1])) 
    result = stub.Predict(request, 10.0) # 10 secs timeout 
    print(result) 

可以在相同載體中傳遞多個圖像以對一批數據運行預測。例如，如果FLAGS.image是逗號分隔的文件名列表：

request = predict_pb2.PredictRequest() 
request.model_spec.name = 'inception' 
request.model_spec.signature_name = 'predict_images' 

# Build a batch of images. 
image_data = [] 
for image in FLAGS.image.split(','): 
    with open(image, 'rb') as f: 
    image_data.append(f.read()) 

request.inputs['images'].CopyFrom(
    tf.contrib.util.make_tensor_proto(image_data, shape=[len(image_data)])) 

result = stub.Predict(request, 10.0) # 10 secs timeout 
print(result) 

if __name__ == '__main__': 
    tf.app.run()