Skip to content

Commit 58d63f3

Browse files
yunyaoXYYyunyaoXYY
authored
[Other] Add detection, segmentation and OCR examples for Ascend deploy. (PaddlePaddle#983)
* Add Huawei Ascend NPU deploy through PaddleLite CANN * Add NNAdapter interface for paddlelite * Modify Huawei Ascend Cmake * Update way for compiling Huawei Ascend NPU deployment * remove UseLiteBackend in UseCANN * Support compile python whlee * Change names of nnadapter API * Add nnadapter pybind and remove useless API * Support Python deployment on Huawei Ascend NPU * Add models suppor for ascend * Add PPOCR rec reszie for ascend * fix conflict for ascend * Rename CANN to Ascend * Rename CANN to Ascend * Improve ascend * fix ascend bug * improve ascend docs * improve ascend docs * improve ascend docs * Improve Ascend * Improve Ascend * Move ascend python demo * Imporve ascend * Improve ascend * Improve ascend * Improve ascend * Improve ascend * Imporve ascend * Imporve ascend * Improve ascend * acc eval script * acc eval * remove acc_eval from branch huawei * Add detection and segmentation examples for Ascend deployment * Add detection and segmentation examples for Ascend deployment * Add PPOCR example for ascend deploy * Imporve paddle lite compiliation * Add FlyCV doc * Add FlyCV doc * Add FlyCV doc * Imporve Ascend docs * Imporve Ascend docs
1 parent 2cfd331 commit 58d63f3

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

57 files changed

+694
-93
lines changed

‎cmake/ascend.cmake

-9
Original file line numberDiff line numberDiff line change
@@ -16,13 +16,4 @@ else ()
1616
if(NOT PADDLELITE_URL)
1717
set(PADDLELITE_URL "https://bj.bcebos.com/fastdeploy/test/lite-linux_arm64_huawei_ascend_npu_python_1207.tgz")
1818
endif()
19-
execute_process(COMMAND sh -c "ls *.so*" WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/third_libs/install/paddlelite/lib
20-
COMMAND sh -c "xargs ${PATCHELF_EXE} --set-rpath '$ORIGIN'" WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/third_libs/install/paddlelite/lib
21-
RESULT_VARIABLE result
22-
OUTPUT_VARIABLE curr_out
23-
ERROR_VARIABLE curr_out)
24-
if(ret EQUAL "1")
25-
message(FATAL_ERROR "Failed to patchelf Paddle Lite libraries when using Ascend.")
26-
endif()
27-
message(STATUS "result:${result} out:${curr_out}")
2819
endif()

‎cmake/paddlelite.cmake

+4
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,10 @@ else()
114114
BUILD_BYPRODUCTS ${PADDLELITE_LIB})
115115
endif()
116116

117+
if(UNIX AND (NOT APPLE) AND (NOT ANDROID))
118+
add_custom_target(patchelf_paddle_lite ALL COMMAND bash -c "PATCHELF_EXE=${PATCHELF_EXE} python ${PROJECT_SOURCE_DIR}/scripts/patch_paddle_lite.py ${PADDLELITE_INSTALL_DIR}/lib/" DEPENDS ${LIBRARY_NAME})
119+
endif()
120+
117121
add_library(external_paddle_lite STATIC IMPORTED GLOBAL)
118122
set_property(TARGET external_paddle_lite PROPERTY IMPORTED_LOCATION ${PADDLELITE_LIB})
119123
add_dependencies(external_paddle_lite ${PADDLELITE_PROJECT})

‎docs/cn/build_and_install/huawei_ascend.md

+8-2
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,12 @@ python setup.py bdist_wheel
9494

9595
#编译完成后,请用户自行安装当前目录的dist文件夹内的whl包.
9696
```
97+
## 五.昇腾部署时开启FlyCV
98+
[FlyCV](https://github.com/PaddlePaddle/FlyCV) 是一款高性能计算机图像处理库, 针对ARM架构做了很多优化, 相比其他图像处理库性能更为出色.
99+
FastDeploy现在已经集成FlyCV, 用户可以在支持的硬件平台上使用FlyCV, 实现模型端到端推理性能的加速.
100+
模型端到端推理中, 预处理和后处理阶段为CPU计算, 当用户使用ARM CPU + 昇腾的硬件平台时, 我们推荐用户使用FlyCV, 可以实现端到端的推理性能加速, 详见[FLyCV使用文档](./boost_cv_by_flycv.md).
97101

98-
- 华为昇腾NPU 上使用C++部署 PaddleClas 分类模型请参考：[PaddleClas 华为升腾NPU C++ 部署示例](../../../examples/vision/classification/paddleclas/ascend/cpp/README.md)
99-
- 华为昇腾NPU 上使用Python部署 PaddleClas 分类模型请参考：[PaddleClas 华为升腾NPU Python 部署示例](../../../examples/vision/classification/paddleclas/ascend/python/README.md)
102+
103+
## 六.昇腾部署Demo参考
104+
- 华为昇腾NPU 上使用C++部署 PaddleClas 分类模型请参考：[PaddleClas 华为升腾NPU C++ 部署示例](../../../examples/vision/classification/paddleclas/cpp/README.md)
105+
- 华为昇腾NPU 上使用Python部署 PaddleClas 分类模型请参考：[PaddleClas 华为升腾NPU Python 部署示例](../../../examples/vision/classification/paddleclas/python/README.md)

‎docs/cn/faq/boost_cv_by_flycv.md

+68
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
[English](../../en/faq/boost_cv_by_flycv.md) | 中文
2+
3+
4+
# 使用FlyCV加速端到端推理性能
5+
6+
[FlyCV](https://github.com/PaddlePaddle/FlyCV) 是一款高性能计算机图像处理库, 针对ARM架构做了很多优化, 相比其他图像处理库性能更为出色.
7+
FastDeploy现在已经集成FlyCV, 用户可以在支持的硬件平台上使用FlyCV, 实现模型端到端推理性能的加速.
8+
9+
## 已支持的系统与硬件架构
10+
11+
| 系统 | 硬件架构 |
12+
| :-----------| :-------- |
13+
| Android | armeabi-v7a, arm64-v8a |
14+
| Linux | aarch64, armhf, x86_64|
15+
16+
17+
## 使用方式
18+
使用FlyCV,首先需要在编译时开启FlyCV编译选项,之后在部署时新增一行代码即可开启.
19+
本文以Linux系统为例,说明如何开启FlyCV编译选项, 之后在部署时, 新增一行代码使用FlyCV.
20+
21+
用户可以按照如下方式,在编译预测库时,开启FlyCV编译选项.
22+
```bash
23+
# 编译C++预测库时, 开启FlyCV编译选项.
24+
-DENABLE_VISION=ON \
25+
26+
# 在编译Python预测库时, 开启FlyCV编译选项
27+
export ENABLE_FLYCV=ON
28+
```
29+
30+
用户可以按照如下方式,在部署代码中新增一行代码启用FlyCV.
31+
```bash
32+
# C++部署代码.
33+
# 新增一行代码启用FlyCV
34+
fastdeploy::vision::EnableFlyCV();
35+
# 其他部署代码...(以昇腾部署为例)
36+
fastdeploy::RuntimeOption option;
37+
option.UseAscend();
38+
...
39+
40+
41+
# Python部署代码
42+
# 新增一行代码启用FlyCV
43+
fastdeploy.vision.enable_flycv()
44+
# 其他部署代码...(以昇腾部署为例)
45+
runtime_option = build_option()
46+
option.use_ascend()
47+
...
48+
```
49+
50+
## 部分平台FlyCV 端到端性能数据
51+
52+
鲲鹏920 CPU + Atlas 300I Pro 推理卡.
53+
| 模型 | OpenCV 端到端性能(ms) | FlyCV 端到端性能(ms) |
54+
| :-----------| :-------- | :-------- |
55+
| ResNet50 | 2.78 | 1.63 |
56+
| PP-LCNetV2 | 2.50 | 1.39 |
57+
| YOLOv7 | 27.00 | 21.36 |
58+
| PP_HumanSegV2_Lite | 2.76 | 2.10 |
59+
60+
61+
瑞芯微RV1126.
62+
63+
| 模型 | OpenCV 端到端性能(ms) | FlyCV 端到端性能(ms) |
64+
| :-----------| :-------- | :-------- |
65+
| ResNet50 | 9.23 | 6.01 |
66+
| mobilenetv1_ssld_量化模型 | 9.23 | 6.01 |
67+
| yolov5s_量化模型 | 28.33 | 14.25 |
68+
| PP_LiteSeg_量化模型 | 132.25 | 60.31 |

‎docs/cn/faq/use_sdk_on_ascend.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
[English](../../en/faq/use_sdk_on_linux.md) | 中文
1+
[English](../../en/faq/use_sdk_on_ascend.md) | 中文
22

33

44
# Linux上使用C++在华为昇腾部署

‎docs/en/build_and_install/huawei_ascend.md

+10-2
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,14 @@ python setup.py bdist_wheel
9393
#After the compilation is complete, please install the whl package in the dist folder of the current directory.
9494
```
9595

96-
Deploying PaddleClas Classification Model on Huawei Ascend NPU using C++ please refer to: [PaddleClas Huawei Ascend NPU C++ Deployment Example](../../../examples/vision/classification/paddleclas/ascend/cpp/README.md)
96+
## Enable FlyCV for Ascend deployment
9797

98-
Deploying PaddleClas classification model on Huawei Ascend NPU using Python please refer to: [PaddleClas Huawei Ascend NPU Python Deployment Example](../../../examples/vision/classification/paddleclas/ascend/python/README.md)
98+
[FlyCV](https://github.com/PaddlePaddle/FlyCV) is a high performance computer image processing library, providing better performance than other image processing libraries, especially in the ARM architecture.
99+
FastDeploy is now integrated with FlyCV, allowing users to use FlyCV on supported hardware platforms to accelerate model end-to-end inference performance.
100+
In end-to-end model inference, the pre-processing and post-processing phases are CPU computation, we recommend using FlyCV for end-to-end inference performance acceleration when you are using ARM CPU + Ascend hardware platform. See [Enable FlyCV](./boost_cv_by_flycv.md) documentation for details.
101+
102+
103+
## Deployment demo reference
104+
- Deploying PaddleClas Classification Model on Huawei Ascend NPU using C++ please refer to: [PaddleClas Huawei Ascend NPU C++ Deployment Example](../../../examples/vision/classification/paddleclas/cpp/README.md)
105+
106+
- Deploying PaddleClas classification model on Huawei Ascend NPU using Python please refer to: [PaddleClas Huawei Ascend NPU Python Deployment Example](../../../examples/vision/classification/paddleclas/python/README.md)

‎docs/en/faq/boost_cv_by_flycv.md

+66
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
[简体中文](../../cn/faq/boost_cv_by_flycv.md) | English
2+
3+
4+
# Accelerate end-to-end inference performance using FlyCV
5+
6+
[FlyCV](https://github.com/PaddlePaddle/FlyCV) is a high performance computer image processing library, providing better performance than other image processing libraries, especially in the ARM architecture.
7+
FastDeploy is now integrated with FlyCV, allowing users to use FlyCV on supported hardware platforms to accelerate model end-to-end inference performance.
8+
9+
## Supported OS and Architectures
10+
11+
| OS | Architectures |
12+
| :-----------| :-------- |
13+
| Android | armeabi-v7a, arm64-v8a |
14+
| Linux | aarch64, armhf, x86_64|
15+
16+
17+
## Usage
18+
To use FlyCV, you first need to turn on the FlyCV compile option at compile time, and then add a new line of code to turn it on.
19+
This article uses Linux as an example to show how to enable the FlyCV compile option, and then add a new line of code to use FlyCV during deployment.
20+
21+
You can turn on the FlyCV compile option when compiling the FastDeploy library as follows.
22+
```bash
23+
# When compiling C++ libraries
24+
-DENABLE_VISION=ON
25+
26+
# When compiling Python libraries
27+
export ENABLE_FLYCV=ON
28+
```
29+
30+
You can enable FlyCV by adding a new line of code to the deployment code as follows.
31+
```bash
32+
# C++ code
33+
fastdeploy::vision::EnableFlyCV();
34+
# Other..(e.g. With Huawei Ascend)
35+
fastdeploy::RuntimeOption option;
36+
option.UseAscend();
37+
...
38+
39+
40+
# Python code
41+
fastdeploy.vision.enable_flycv()
42+
# Other..(e.g. With Huawei Ascend)
43+
runtime_option = build_option()
44+
option.use_ascend()
45+
...
46+
```
47+
48+
## Some Platforms FlyCV End-to-End Inference Performance
49+
50+
KunPeng 920 CPU + Atlas 300I Pro.
51+
| Model | OpenCV E2E Performance(ms) | FlyCV E2E Performance(ms) |
52+
| :-----------| :-------- | :-------- |
53+
| ResNet50 | 2.78 | 1.63 |
54+
| PP-LCNetV2 | 2.50 | 1.39 |
55+
| YOLOv7 | 27.00 | 21.36 |
56+
| PP_HumanSegV2_Lite | 2.76 | 2.10 |
57+
58+
59+
Rockchip RV1126.
60+
61+
| Model | OpenCV E2E Performance(ms) | FlyCV E2E Performance(ms) |
62+
| :-----------| :-------- | :-------- |
63+
| ResNet50 | 9.23 | 6.01 |
64+
| mobilenetv1_ssld_量化模型 | 9.23 | 6.01 |
65+
| yolov5s_量化模型 | 28.33 | 14.25 |
66+
| PP_LiteSeg_量化模型 | 132.25 | 60.31 |

‎docs/en/faq/use_sdk_on_ascend.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
[简体中文](../../cn/faq/use_sdk_on_linux.md) | English
1+
[简体中文](../../cn/faq/use_sdk_on_ascend.md) | English
22

33

44
# # Linux deployment with C++ on Huawei Ascend

‎examples/vision/detection/paddledetection/cpp/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ tar xvf ppyoloe_crn_l_300e_coco.tgz
3434
./infer_ppyoloe_demo ./ppyoloe_crn_l_300e_coco 000000014439.jpg 2
3535
# 昆仑芯XPU推理
3636
./infer_ppyoloe_demo ./ppyoloe_crn_l_300e_coco 000000014439.jpg 3
37+
# 华为昇腾推理
38+
./infer_ppyoloe_demo ./ppyoloe_crn_l_300e_coco 000000014439.jpg 4
3739
```
3840

3941
以上命令只适用于Linux或MacOS, Windows下SDK的使用方式请参考:

‎examples/vision/detection/paddledetection/cpp/infer_ppyolo.cc

+29
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,33 @@ void GpuInfer(const std::string& model_dir, const std::string& image_file) {
102102
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
103103
}
104104

105+
void AscendInfer(const std::string& model_dir, const std::string& image_file) {
106+
auto model_file = model_dir + sep + "model.pdmodel";
107+
auto params_file = model_dir + sep + "model.pdiparams";
108+
auto config_file = model_dir + sep + "infer_cfg.yml";
109+
auto option = fastdeploy::RuntimeOption();
110+
option.UseAscend();
111+
auto model = fastdeploy::vision::detection::PPYOLO(model_file, params_file,
112+
config_file, option);
113+
if (!model.Initialized()) {
114+
std::cerr << "Failed to initialize." << std::endl;
115+
return;
116+
}
117+
118+
auto im = cv::imread(image_file);
119+
120+
fastdeploy::vision::DetectionResult res;
121+
if (!model.Predict(im, &res)) {
122+
std::cerr << "Failed to predict." << std::endl;
123+
return;
124+
}
125+
126+
std::cout << res.Str() << std::endl;
127+
auto vis_im = fastdeploy::vision::VisDetection(im, res, 0.5);
128+
cv::imwrite("vis_result.jpg", vis_im);
129+
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
130+
}
131+
105132
int main(int argc, char* argv[]) {
106133
if (argc < 4) {
107134
std::cout
@@ -120,6 +147,8 @@ int main(int argc, char* argv[]) {
120147
GpuInfer(argv[1], argv[2]);
121148
} else if (std::atoi(argv[3]) == 2) {
122149
KunlunXinInfer(argv[1], argv[2]);
150+
} else if (std::atoi(argv[3]) == 3) {
151+
AscendInfer(argv[1], argv[2]);
123152
}
124153
return 0;
125154
}

‎examples/vision/detection/paddledetection/cpp/infer_ppyoloe.cc

+29
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,33 @@ void TrtInfer(const std::string& model_dir, const std::string& image_file) {
131131
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
132132
}
133133

134+
void AscendInfer(const std::string& model_dir, const std::string& image_file) {
135+
auto model_file = model_dir + sep + "model.pdmodel";
136+
auto params_file = model_dir + sep + "model.pdiparams";
137+
auto config_file = model_dir + sep + "infer_cfg.yml";
138+
auto option = fastdeploy::RuntimeOption();
139+
option.UseAscend();
140+
auto model = fastdeploy::vision::detection::PPYOLOE(model_file, params_file,
141+
config_file, option);
142+
if (!model.Initialized()) {
143+
std::cerr << "Failed to initialize." << std::endl;
144+
return;
145+
}
146+
147+
auto im = cv::imread(image_file);
148+
149+
fastdeploy::vision::DetectionResult res;
150+
if (!model.Predict(im, &res)) {
151+
std::cerr << "Failed to predict." << std::endl;
152+
return;
153+
}
154+
155+
std::cout << res.Str() << std::endl;
156+
auto vis_im = fastdeploy::vision::VisDetection(im, res, 0.5);
157+
cv::imwrite("vis_result.jpg", vis_im);
158+
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
159+
}
160+
134161
int main(int argc, char* argv[]) {
135162
if (argc < 4) {
136163
std::cout
@@ -151,6 +178,8 @@ int main(int argc, char* argv[]) {
151178
TrtInfer(argv[1], argv[2]);
152179
} else if (std::atoi(argv[3]) == 3) {
153180
KunlunXinInfer(argv[1], argv[2]);
181+
} else if (std::atoi(argv[3]) == 4) {
182+
AscendInfer(argv[1], argv[2]);
154183
}
155184
return 0;
156185
}

‎examples/vision/detection/paddledetection/cpp/infer_ssd.cc

+29
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,33 @@ void GpuInfer(const std::string& model_dir, const std::string& image_file) {
104104
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
105105
}
106106

107+
void AscendInfer(const std::string& model_dir, const std::string& image_file) {
108+
auto model_file = model_dir + sep + "model.pdmodel";
109+
auto params_file = model_dir + sep + "model.pdiparams";
110+
auto config_file = model_dir + sep + "infer_cfg.yml";
111+
auto option = fastdeploy::RuntimeOption();
112+
option.UseAscend();
113+
auto model = fastdeploy::vision::detection::SSD(model_file, params_file,
114+
config_file, option);
115+
if (!model.Initialized()) {
116+
std::cerr << "Failed to initialize." << std::endl;
117+
return;
118+
}
119+
120+
auto im = cv::imread(image_file);
121+
122+
fastdeploy::vision::DetectionResult res;
123+
if (!model.Predict(im, &res)) {
124+
std::cerr << "Failed to predict." << std::endl;
125+
return;
126+
}
127+
128+
std::cout << res.Str() << std::endl;
129+
auto vis_im = fastdeploy::vision::VisDetection(im, res, 0.5);
130+
cv::imwrite("vis_result.jpg", vis_im);
131+
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
132+
}
133+
107134
int main(int argc, char* argv[]) {
108135
if (argc < 4) {
109136
std::cout
@@ -122,6 +149,8 @@ int main(int argc, char* argv[]) {
122149
GpuInfer(argv[1], argv[2]);
123150
} else if (std::atoi(argv[3]) == 2) {
124151
KunlunXinInfer(argv[1], argv[2]);
152+
} else if (std::atoi(argv[3]) == 3) {
153+
AscendInfer(argv[1], argv[2]);
125154
}
126155
return 0;
127156
}

‎examples/vision/detection/paddledetection/cpp/infer_yolov3.cc

+30
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,34 @@ void GpuInfer(const std::string& model_dir, const std::string& image_file) {
102102
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
103103
}
104104

105+
106+
void AscendInfer(const std::string& model_dir, const std::string& image_file) {
107+
auto model_file = model_dir + sep + "model.pdmodel";
108+
auto params_file = model_dir + sep + "model.pdiparams";
109+
auto config_file = model_dir + sep + "infer_cfg.yml";
110+
auto option = fastdeploy::RuntimeOption();
111+
option.UseAscend();
112+
auto model = fastdeploy::vision::detection::YOLOv3(model_file, params_file,
113+
config_file, option);
114+
if (!model.Initialized()) {
115+
std::cerr << "Failed to initialize." << std::endl;
116+
return;
117+
}
118+
119+
auto im = cv::imread(image_file);
120+
121+
fastdeploy::vision::DetectionResult res;
122+
if (!model.Predict(im, &res)) {
123+
std::cerr << "Failed to predict." << std::endl;
124+
return;
125+
}
126+
127+
std::cout << res.Str() << std::endl;
128+
auto vis_im = fastdeploy::vision::VisDetection(im, res, 0.5);
129+
cv::imwrite("vis_result.jpg", vis_im);
130+
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
131+
}
132+
105133
int main(int argc, char* argv[]) {
106134
if (argc < 4) {
107135
std::cout
@@ -120,6 +148,8 @@ int main(int argc, char* argv[]) {
120148
GpuInfer(argv[1], argv[2]);
121149
} else if (std::atoi(argv[3]) == 2) {
122150
KunlunXinInfer(argv[1], argv[2]);
151+
} else if (std::atoi(argv[3]) == 3) {
152+
AscendInfer(argv[1], argv[2]);
123153
}
124154
return 0;
125155
}

‎examples/vision/detection/paddledetection/python/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,8 @@ python infer_ppyoloe.py --model_dir ppyoloe_crn_l_300e_coco --image 000000014439
2525
python infer_ppyoloe.py --model_dir ppyoloe_crn_l_300e_coco --image 000000014439.jpg --device gpu --use_trt True
2626
# 昆仑芯XPU推理
2727
python infer_ppyoloe.py --model_dir ppyoloe_crn_l_300e_coco --image 000000014439.jpg --device kunlunxin
28+
# 华为昇腾推理
29+
python infer_ppyoloe.py --model_dir ppyoloe_crn_l_300e_coco --image 000000014439.jpg --device ascend
2830
```
2931

3032
运行完成可视化结果如下图所示

0 commit comments

Comments
 (0)