[原创] 一个简单的TensorFlow-Serving例子

本文展示了如何用Python训练一个简单的神经网络模型,保存为模型文件,并且用TensorFlow-Serving的Docker镜像把它加载起来,提供在线服务的过程。
环境:Ubuntu 16.04 LTS,TensorFlow 1.14.0,Python 3.6.8

[×] 训练一个简单的神经网络模型
直接看代码:

import os
import shutil
 
import numpy as np
import tensorflow as tf
 
 
def add_layer(inputs, input_size, output_size, activation_function=None):
    weights = tf.Variable(tf.random_normal([input_size, output_size]))
    biases = tf.Variable(tf.zeros([1, output_size]) + 0.1)
    wx_plus_b = tf.matmul(inputs, weights) + biases  # WX + b
    if activation_function is None:
        outputs = wx_plus_b
    else:
        outputs = activation_function(wx_plus_b)
    return outputs
 
 
# 造一些随机输入数据
num_points = 30000  # 总数据条数
feature_number = 100  # 每条输入数据有100个feature
# num_points个输入数据,每个有feature_number个feature,即输入数据的维度是(num_points,feature_number)
x_data = np.random.rand(num_points, feature_number)
y_data = np.random.randint(0, 2, (num_points, 1))  # nx1的数组, 每一行为1个数(0或1)
 
# 用于接收输入的Tensor
x_actual = tf.placeholder(tf.float32, [None, feature_number], name="myInput")
y_actual = tf.placeholder(tf.float32, [None, 1], name="myOutput")
 
# 隐层1
l1 = add_layer(x_actual, feature_number, 32, activation_function=tf.nn.relu)
# 隐层2
l2 = add_layer(l1, 32, 64, activation_function=tf.nn.tanh)
# 隐层3
l3 = add_layer(l2, 64, 32, activation_function=tf.nn.relu)
 
# 输出层
y_predict = add_layer(l3, 32, 1, activation_function=tf.nn.sigmoid)
# 损失函数
loss = -tf.reduce_mean(y_actual * tf.log(tf.clip_by_value(y_predict, 1e-10, 1.0)))
# 优化器
train_step = tf.train.AdamOptimizer(0.001).minimize(loss)
 
init = tf.global_variables_initializer()
# 迭代次数
num_iterations = 10000
with tf.Session() as sess:
    sess.run(init)
    for i in range(num_iterations):
        # 训练模型
        sess.run(train_step, feed_dict={x_actual: x_data, y_actual: y_data})
        if i % 500 == 0:
            prediction_value = sess.run(y_predict, feed_dict={x_actual: x_data})
            print(sess.run(loss, feed_dict={x_actual: x_data, y_actual: y_data}))
    # 训练完成后,以SavedModel格式保存模型文件
    model_output_dir = "./model/201908070001"
    if os.path.exists(model_output_dir):  # 目录存在
        shutil.rmtree(model_output_dir)  # 删除原目录
    tf.saved_model.simple_save(
        sess, model_output_dir, inputs={"myInput": x_actual}, outputs={"myOutput": y_predict})
 
    # 做5次预测(测试一下)
    for i in range(5):
        x_input = np.random.rand(1, feature_number)  # 1表示输入一条数据
        feed_dict = {x_actual: x_input}
        result = sess.run(y_predict, feed_dict)
        print('prediction result: %f' % result)

(据反映,有些人的浏览器看到上面的代码是没有正确缩进的,可以自己复制到 IDE 里处理一下)
数据是随机生成的,模型是随意定义的,所以不要在意其合理性。
模型要以SavedModel格式保存,否则不能用TensorFlow-Serving来serve,在这里我输出的模型目录为./model/201908070001,里面的文件大概有这些:

├── saved_model.pb
└── variables
    ├── variables.data-00000-of-00001
    └── variables.index

文章来源:https://www.codelast.com/
[×] 准备TensorFlow-Serving的Docker镜像
首先你当然得安装Docker(略),然后下载TF-Serving的Docker镜像:

docker pull tensorflow/serving

下载完后,再 cd 到上面输出的“model”目录的上一级目录:

TESTDATA="$(pwd)/model"
docker run -t --rm -p 8501:8501 \
    -v "$TESTDATA:/models/simple_fc_nn" \
    -e MODEL_NAME=simple_fc_nn \
    tensorflow/serving
这样就可以把模型serve起来了。其中,端口号可以自己改,simple_fc_nn是我自己起的模型名称,在后面使用REST API来访问TF-Serving服务的时候,会用到这个名称。
文章来源:https://www.codelast.com/
正常的话,命令行会输出类似于下面的日志:

tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config:  model_name: simple_fc_nn model_base_path: /models/simple_fc_nn
tensorflow_serving/model_servers/server_core.cc:462] Adding/updating models.
tensorflow_serving/model_servers/server_core.cc:561]  (Re-)adding model: simple_fc_nn
tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: simple_fc_nn version: 201908070001}
tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: simple_fc_nn version: 201908070001}
tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: simple_fc_nn version: 201908070001}
external/org_tensorflow/tensorflow/contrib/session_bundle/bundle_shim.cc:363] Attempting to load native SavedModelBundle in bundle-shim from: /models/simple_fc_nn/201908070001
external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /models/simple_fc_nn/201908070001
external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:202] Restoring SavedModel bundle.
external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:311] SavedModel load for tags { serve }; Status: success. Took 57164 microseconds.
tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:103] No warmup data file found at /models/simple_fc_nn/201908070001/assets.extra/tf_serving_warmup_requests
tensorflow_serving/core/loader_harness.cc:86] Successfully loaded servable version {name: simple_fc_nn version: 201908070001}
tensorflow_serving/model_servers/server.cc:324] Running gRPC ModelServer at 0.0.0.0:8500 ...
[warn] getaddrinfo: address family for nodename not supported
[evhttp_server.cc : 239] RAW: Entering the event loop ...
tensorflow_serving/model_servers/server.cc:344] Exporting HTTP/REST API at:localhost:8501 ...
此时,我们就可以通过REST API来访问这个服务,来进行模型预测了。
文章来源:https://www.codelast.com/
[×] 通过REST API查看服务状态
curl http://localhost:8501/v1/models/simple_fc_nn
这里可以看到,URL里的simple_fc_nn就是在启动TF-Serving服务的时候指定的那个名字。
另外,这里使用的是localhost,所以必须在TF-Serving运行的同一台机器上执行该命令。
返回:

{

   "model_version_status": [

       {

           "version": "201908070001",

           "state": "AVAILABLE",

           "status": {

               "error_code": "OK",

               "error_message": ""

           }

       }

   ]

}

[×] 通过REST API查看模型的元数据

curl http://localhost:8501/v1/models/simple_fc_nn/metadata

返回:

{

   "model_spec": {

       "name": "simple_fc_nn",

       "signature_name": "",

       "version": "201908070001"

   },

   "metadata": {

       "signature_def": {

           "signature_def": {

               "serving_default": {

                   "inputs": {

                       "myInput": {

                           "dtype": "DT_FLOAT",

                           "tensor_shape": {

                               "dim": [

                                   {

                                       "size": "-1",

                                       "name": ""

                                   },

                                   {

                                       "size": "100",

                                       "name": ""

                                   }

                               ],

                               "unknown_rank": false

                           },

                           "name": "myInput:0"

                       }

                   },

                   "outputs": {

                       "myOutput": {

                           "dtype": "DT_FLOAT",

                           "tensor_shape": {

                               "dim": [

                                   {

                                       "size": "-1",

                                       "name": ""

                                   },

                                   {

                                       "size": "1",

                                       "name": ""

                                   }

                               ],

                               "unknown_rank": false

                           },

                           "name": "Sigmoid:0"

                       }

                   },

                   "method_name": "tensorflow/serving/predict"

               }

           }

       }

   }

}

文章来源:https://www.codelast.com/
 

[×] 通过REST API做一次模型预测

curl -d '{"instances": [[0.9255854, 0.6900963, 0.16456964, 0.83122249, 0.53394498, 0.0966489, 0.33756461, 0.39532023, 0.90302752, 0.56739237, 0.73394432, 0.85927172, 0.37721589, 0.54909182, 0.46008562, 0.96891122, 0.62722268, 0.06291056, 0.58401942, 0.16767831, 0.94320249, 0.22090671, 0.65477176, 0.1654681, 0.02658432, 0.21340836, 0.01726125, 0.34000626, 0.51242977, 0.8792962, 0.30855666, 0.77404037, 0.92957236, 0.12984285, 0.28480676, 0.20054448, 0.88745966, 0.89472512, 0.70519433, 0.52899148, 0.88639978, 0.45279248, 0.55500912, 0.74058798, 0.92649993, 0.65843703, 0.27740498, 0.3078109, 0.39115213, 0.67084904, 0.00103263, 0.19882637, 0.69135604, 0.98284994, 0.56733945, 0.14392018, 0.75021845, 0.69635296, 0.94257055, 0.16369508, 0.07440702, 0.86413908, 0.19726159, 0.59199729, 0.56901319, 0.92222904, 0.73738314, 0.08026704, 0.84337852, 0.00719291, 0.77943135, 0.47079168, 0.12110838, 0.8664636, 0.41693313, 0.50308834, 0.19198665, 0.25000135, 0.2613975, 0.48228129, 0.2522098, 0.08535753, 0.86548783, 0.74076042, 0.80560582, 0.33183196, 0.64742734, 0.71139366, 0.13395337, 0.90973808, 0.68171421, 0.5592008, 0.46866331, 0.58312591, 0.71402775, 0.82166998, 0.3564542, 0.97947134, 0.23538156, 0.9138861]]}' -X POST http://localhost:8501/v1/models/simple_fc_nn:predict

由于训练模型的时候定义了输入的每条数据为一个100维的向量,因此我造了上面的数据。
返回:

{"predictions": [[1]]}

这就算完成了一次预测过程。
文章来源:https://www.codelast.com/

[×] 通过Apache ab对TF-Serving进行性能测试

ab -n 100000 -c 50 -T 'Content-Type:application/json' -p ./post.txt http://localhost:8501/v1/models/simple_fc_nn:predict

其中,post.txt 保存的是上面请求中的JSON字符串。
ab会打印出详细的benchmark数据,包括QPS等:

Document Path:          /v1/models/simple_fc_nn:predict

Document Length:        42 bytes

 

Concurrency Level:      50

Time taken for tests:   7.424 seconds

Complete requests:      100000

Failed requests:        0

Total transferred:      9300000 bytes

Total body sent:        138500000

HTML transferred:       4200000 bytes

Requests per second:    13469.99 [#/sec] (mean)

Time per request:       3.712 [ms] (mean)

Time per request:       0.074 [ms] (mean, across all concurrent requests)

Transfer rate:          1223.35 [Kbytes/sec] received

                        18218.68 kb/s sent

                        19442.03 kb/s total

 

Connection Times (ms)

              min  mean[+/-sd] median   max

Connect:        0 1 0.3   0 4

Processing:     1 3 1.2   3 211

Waiting:        1 3 1.2   3 211

Total:          1 4 1.2   4 212

文章来源:https://www.codelast.com/
[×] 停掉TF-Serving服务
先找到Docker容器进程:

docker container ps | grep "tensorflow/serving"

输出:

2bc9547d56b0        tensorflow/serving   "/usr/bin/tf_serving…"   18 minutes ago      Up 18 minutes       8500/tcp, 0.0.0.0:8501->8501/tcp   strange_hopper

第一列为container id,干掉它即可:

docker container kill 2bc9547d56b0

(完)。
文章来源:https://www.codelast.com/
➤➤ 版权声明 ➤➤ 
转载需注明出处:codelast.com 
感谢关注我的微信公众号(微信扫一扫):

wechat qrcode of codelast

发表评论