6-5使用tensorflow-serving部署模型——eat_tensorflow2_in_30_days
作者：互联网
6-5使用tensorflow-serving部署模型
TensorFlow训练好的模型以tensorflow原生方式保存成protobuf文件后可以用许多方式部署运行。
例如：通过 tensorflow-js 可以用javascrip脚本加载模型并在浏览器中运行模型。
通过 tensorflow-lite 可以在移动和嵌入式设备上加载并运行TensorFlow模型。
通过 tensorflow-serving 可以加载模型后提供网络接口API服务，通过任意编程语言发送网络请求都可以获取模型预测结果。
通过 tensorFlow for Java接口，可以在Java或者spark(scala)中调用tensorflow模型进行预测。
我们主要介绍tensorflow serving部署模型、使用spark(scala)调用tensorflow模型的方法。
tensorflow serving模型部署概述
使用 tensorflow serving 部署模型要完成以下步骤。
- (1) 准备protobuf模型文件。
- (2) 安装tensorflow serving。
- (3) 启动tensorflow serving 服务。
- (4) 向API服务发送请求，获取预测结果。
准备protobuf模型文件
- 我们使用tf.keras 训练一个简单的线性回归模型，并保存成protobuf文件。
import tensorflow as tf
from tensorflow.keras import models,layers,optimizers
## 样本数量
n = 800
## 生成测试用数据集
X = tf.random.uniform([n,2],minval=-10,maxval=10)
w0 = tf.constant([[2.0],[-1.0]])
b0 = tf.constant(3.0)
Y = X@w0 + b0 + tf.random.normal([n,1],
mean = 0.0,stddev= 2.0) # @表示矩阵乘法,增加正态扰动
## 建立模型
tf.keras.backend.clear_session()
inputs = layers.Input(shape = (2,),name ="inputs") #设置输入名字为inputs
outputs = layers.Dense(1, name = "outputs")(inputs) #设置输出名字为outputs
linear = models.Model(inputs = inputs,outputs = outputs)
linear.summary()
## 使用fit方法进行训练
linear.compile(optimizer="rmsprop",loss="mse",metrics=["mae"])
linear.fit(X,Y,batch_size = 8,epochs = 100)
tf.print("w = ",linear.layers[1].kernel)
tf.print("b = ",linear.layers[1].bias)
## 将模型保存成pb格式文件
export_path = "./data/linear_model/"
version = "1" #后续可以通过版本号进行模型版本迭代与管理
linear.save(export_path+version, save_format="tf")
"""
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
inputs (InputLayer) [(None, 2)] 0
_________________________________________________________________
outputs (Dense) (None, 1) 3
=================================================================
Total params: 3
Trainable params: 3
Non-trainable params: 0
_________________________________________________________________
INFO:tensorflow:Assets written to: ./data/linear_model/1/assets
"""
#查看保存的模型文件
!ls {export_path+version}
"""
assets keras_metadata.pb saved_model.pb variables
"""
# 查看模型文件相关信息
!saved_model_cli show --dir {export_path+str(version)} --all
"""
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['__saved_model_init_op']:
The given SavedModel SignatureDef contains the following input(s):
The given SavedModel SignatureDef contains the following output(s):
outputs['__saved_model_init_op'] tensor_info:
dtype: DT_INVALID
shape: unknown_rank
name: NoOp
Method name is:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 2)
name: serving_default_inputs:0
The given SavedModel SignatureDef contains the following output(s):
outputs['outputs'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 1)
name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict
Concrete Functions:
Function Name: '__call__'
Option #1
Callable with:
Argument #1
inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name='inputs')
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #2
Callable with:
Argument #1
inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name='inputs')
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
Function Name: '_default_save_signature'
Option #1
Callable with:
Argument #1
inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name='inputs')
Function Name: 'call_and_return_all_conditional_losses'
Option #1
Callable with:
Argument #1
inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name='inputs')
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #2
Callable with:
Argument #1
inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name='inputs')
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
"""
安装 tensorflow serving
安装 tensorflow serving 有2种主要方法：通过Docker镜像安装，通过apt安装。
通过Docker镜像安装是最简单，最直接的方法，推荐采用。
Docker可以理解成一种容器，其上面可以给各种不同的程序提供独立的运行环境。
一般业务中用到tensorflow的企业都会有运维同学通过Docker 搭建 tensorflow serving.
无需算法工程师同学动手安装，以下安装过程仅供参考。
不同操作系统机器上安装Docker的方法可以参照以下链接。
https://www.runoob.com/docker/ubuntu-docker-install.html
安装Docker成功后，使用如下命令加载 tensorflow/serving 镜像到Docker中
docker pull tensorflow/serving
这里先演示不基于Docker安装的TensorFlow serving
!curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | apt-key add -
"""
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2943 100 2943 0 0 15248 0 --:--:-- --:--:-- --:--:-- 15248
OK
"""
# 命令行切换root用户
curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | apt-key add -
apt update
echo "deb http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | tee /etc/apt/sources.list.d/tensorflow-serving.list && curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | apt-key add -
apt update
apt-get install tensorflow-model-server
启动 tensorflow serving 服务
# 必需绝对路径
import os
os.environ["MODEL_DIR"] = "/media/ps/BigVolumnDisk/learning/learning_tensorflow/eat_tensorflow2_in_30_days_ipynb-master/六、TensorFlow的高阶API/data/linear_model/"
os.environ["MODEL_DIR"]
!ls ./data/linear_model/
"""
1
"""
%%bash --bg
nohup tensorflow_model_server \
--rest_api_port=8501 \
--model_name=linear_model \
--model_base_path="${MODEL_DIR}" >server.log 2>&1
# 查看日志，有可能会有报错信息
!tail server.log
"""
2022-07-02 19:11:38.916661: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:212] Running initialization op on SavedModel bundle at path: /media/ps/BigVolumnDisk/learning/learning_tensorflow/eat_tensorflow2_in_30_days_ipynb-master/六、TensorFlow的高阶API/data/linear_model/1
2022-07-02 19:11:38.920029: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:301] SavedModel load for tags { serve }; Status: success: OK. Took 70823 microseconds.
2022-07-02 19:11:38.920183: I tensorflow_serving/servables/tensorflow/saved_model_warmup_util.cc:59] No warmup data file found at /media/ps/BigVolumnDisk/learning/learning_tensorflow/eat_tensorflow2_in_30_days_ipynb-master/六、TensorFlow的高阶API/data/linear_model/1/assets.extra/tf_serving_warmup_requests
2022-07-02 19:11:38.920295: I tensorflow_serving/core/loader_harness.cc:95] Successfully loaded servable version {name: linear_model version: 1}
2022-07-02 19:11:38.922644: I tensorflow_serving/model_servers/server_core.cc:486] Finished adding/updating models
2022-07-02 19:11:38.922673: I tensorflow_serving/model_servers/server.cc:133] Using InsecureServerCredentials
2022-07-02 19:11:38.922677: I tensorflow_serving/model_servers/server.cc:395] Profiler service is enabled
2022-07-02 19:11:38.922925: I tensorflow_serving/model_servers/server.cc:421] Running gRPC ModelServer at 0.0.0.0:8500 ...
2022-07-02 19:11:38.924267: I tensorflow_serving/model_servers/server.cc:442] Exporting HTTP/REST API at:localhost:8501 ...
[evhttp_server.cc : 245] NET_LOG: Entering the event loop ...
"""
ps -ef | grep tensorflow_model_server
# """
# ps 246227 246225 0 19:11 ? 00:00:00 tensorflow_model_server --rest_api_port=8501 --model_name=linear_model --model_base_path=/media/ps/BigVolumnDisk/learning/learning_tensorflow/eat_tensorflow2_in_30_days_ipynb-master/六、TensorFlow的高阶API/data/linear_model/
# root 246488 244905 0 19:12 pts/1 00:00:00 grep --color=auto tensorflow_model_server
# """
向服务API发送请求
可以使用任何编程语言的http功能发送请求，下面示范linux的 curl 命令发送请求，以及Python的requests库发送请求。
使用 Linux 的curl命令发送请求
!curl -d '{"instances": [[1.0, 2.0], [5.0, 8.0]]}' -X POST http://localhost:8501/v1/models/linear_model:predict
"""
{
"predictions": [[2.85995507], [4.81792879]
]
}
"""
# 利用 Python request库发送请求
import json
import requests
data = json.dumps({"signature_name": "serving_default", "instances": [[1.0, 2.0], [5.0,7.0]]})
headers = {"content-type": "application/json"}
json_response = requests.post('http://localhost:8501/v1/models/linear_model:predict', data=data, headers=headers)
predictions = json.loads(json_response.text)["predictions"]
print(predictions)
"""
[[2.85995507], [5.8326869]]
"""
标签：tensorflow2,inputs,serving,linear,--,30,tensorflow,model 来源： https://www.cnblogs.com/lotuslaw/p/16438398.html