Moving Out Of Vivante NPU Demo Docker Container
The awnn demo package has yolo11 demo as such,
radxa@radxa-cubie-a7z:~$ cd awnpu_model_zoo-v1.0.0-20260423-f562dd16/examples/yolo11/install/yolo11_demo_linux_a733/ $ export LD_LIBRARY_PATH=../../../../common/npuruntime/lib_linux_aarch64/A733/:$LD_LIBRARY_PATH $ ls -l /dev/vipcore crw-rw-rw- 1 root root 199, 0 Jun 28 04:55 /dev/vipcore $ ./yolo11_demo_a733 -nb model/yolo11s_6_uint8_a733.nb -i model/dog.jpg model_file=model/yolo11s_6_uint8_a733.nb, input=model/dog.jpg, loop_count=1, malloc_mbyte=10 VIPLite driver software version 2.0.3.2-AW-2024-08-30 input 0 dim 3 640 640 1, data_format=2, quant_format=0, name=input/output[0], none-quant output 0 dim 80 80 64 1, data_format=0, name=uid_11_out_0b_uid_1_out_0, none-quant output 1 dim 80 80 80 1, data_format=0, name=uid_10_out_0b_uid_1_out_0, none-quant output 2 dim 40 40 64 1, data_format=0, name=uid_9_out_0ub_uid_1_out_0, none-quant output 3 dim 40 40 80 1, data_format=0, name=uid_8_out_0ub_uid_1_out_0, none-quant output 4 dim 20 20 64 1, data_format=0, name=uid_7_out_0ub_uid_1_out_0, none-quant output 5 dim 20 20 80 1, data_format=0, name=uid_6_out_0ub_uid_1_out_0, none-quant nbg name=model/yolo11s_6_uint8_a733.nb, size: 6850488. create network 0: 9417 us. prepare network: 2358 us. buffer ptr: 0xb186600, buffer size: 1228800 network: 0, loop count: 1 run time for this network 0: 31966 us. output 0, ptr 0xb2b2740, size 409600. output 1, ptr 0xb4427c0, size 512000. output 2, ptr 0xb636840, size 102400. output 3, ptr 0xb69a8c0, size 128000. output 4, ptr 0xb717980, size 25600. output 5, ptr 0xb730a00, size 32000. post process time : 5 ms detection num: 3 1: 94%, [ 127, 129, 569, 419], bicycle 16: 92%, [ 132, 220, 311, 541], dog 2: 49%, [ 466, 75, 692, 171], car destory npu finished. ~NpuUint. radxa@radxa-cubie-a7z:~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/examples/yolo11/install/yolo11_demo_linux_a733$
TODO: With the publicly available Shahed loiter munition YOLO model, similar to https://github.com/OpenIPC/waybeam_venc, we need to integrate converged_aiot's fcu_thread with detected object screen coordinates, such as the above bicycle at [127,129,569,419], to steer the craft onto the recognized Shahed's trajectory. But there is troubleshooting to do to merge yolo11_demo_linux_a733's source code:
Our converged_aiot, via GStreamer code, indirectly uses OpenCV in the Radxa R6 image, but is out of sync with OpenCV in the Radxa NPU "Model Zoo" YOLO demo code, which includes a local OpenCV version in the demo folder. So, we need to modify the building procedure build_linux.sh of the Model Zoo's C++ demo code, for which we are not given the building g++ command lines as such,
radxa@radxa-cubie-a7z:~/awnpu_model_zoo-v1.0.0-20260423-f562dd16$ grep -r vip_lite.h * common/npuruntime/npu_util.cpp:#include <vip_lite.h> common/npuruntime/npulib.cpp:#include <vip_lite.h> functions/vpm_run/src/parse_sample_txt.cpp:#include <vip_lite.h> radxa@radxa-cubie-a7z:~/awnpu_model_zoo-v1.0.0-20260423-f562dd16$ ... root@104012560a4f:/workspace/examples/yolo11# ../build_linux.sh -t a733 -s debian11 Option TARGET_NAME with argument: a733 build TARGET=a733 =========== check target ok =========== Option TARGET_SYSTEM with argument: debian11 -- cur dir of *.cmake: /workspace/cmake_toolchain -- cur dir of *.cmake: /workspace/cmake_toolchain -- The C compiler identification is GNU 10.2.1 -- The CXX compiler identification is GNU 10.2.1 -- Check for working C compiler: /workspace/cmake_toolchain/../0-toolchains/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/ -- Check for working C compiler: /workspace/cmake_toolchain/../0-toolchains/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/ -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /workspace/cmake_toolchain/../0-toolchains/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/ -- Check for working CXX compiler: /workspace/cmake_toolchain/../0-toolchains/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/ -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- USE EXTERN DEFINE TARGET -- USE EXTERN TOOLCHAIN BUILD -- CMAKE_C_COMPILER = /workspace/cmake_toolchain/../0-toolchains/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin/aarch64 -- CMAKE_SYSTEM_NAME = Linux -- TARGET_NAME = A733 -- Found OpenCV: /workspace/3rdparty/opencv/opencv-4.9.0-aarch64-linux-sunxi-glibc (found version "4.9.0") -- CMAKE_INSTALL_PREFIX = /workspace/examples/yolo11/install/yolo11_demo_linux_a733 -- Configuring done -- Generating done -- Build files have been written to: /workspace/examples/yolo11/build_linux_aarch64 Scanning dependencies of target yolo11_demo_a733 [ 16%] Building CXX object CMakeFiles/yolo11_demo_a733.dir/main.cpp.o [ 33%] Building CXX object CMakeFiles/yolo11_demo_a733.dir/yolo11_6_post.cpp.o [ 50%] Building CXX object CMakeFiles/yolo11_demo_a733.dir/yolo11_6_pre.cpp.o [ 66%] Building CXX object CMakeFiles/yolo11_demo_a733.dir/workspace/common/npuruntime/npu_util.cpp.o [ 83%] Building CXX object CMakeFiles/yolo11_demo_a733.dir/workspace/common/npuruntime/npulib.cpp.o [100%] Linking CXX executable yolo11_demo_a733Fortunately, we are given a glimpse of the "cc" command line to compile and build with vip_lite.h API with Radxa's ai-sdk demo program vpm_run, as so,
radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ make install AI_SDK_PLATFORM=a733 INSTALL_PREFIX=./
cc -g -O0 -DDEBUG -D_DEBUG -DSAVE_OUTPUT_TXT_FILE -DSHOW_TOP5 -DNPU_SW_VERSION=2 -g -o vpm_run vpm_run.c
-I.//usr/include -I../libawnn_viplite -I../libawutils -I../../viplite-tina/lib/aarch64-none-linux-gnu/v2.0/inc
-L .//usr/lib -Wl,-rpath-link,.//usr/lib -L../../viplite-tina/lib/aarch64-none-linux-gnu/v2.0/ -lNBGlinker -lVIPhal -lm
radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ grep vip_lite.h vpm_run.c
#include <vip_lite.h>
radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ cd etc/npu/vpm_run/
radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run/etc/npu/vpm_run$ ./vpm_run -s sample.txt -l 1
loop_count=1, device_index=0, core_index=-1, file_name=sample.txt, time_out=0x0, bypass=1
enable_npd=0, preload=0
show_top50, save_txt=0
init vip lite, driver version=0x00020003...
VIPLite driver software version 2.0.3.2-AW-2024-08-30
vip lite init OK.
cid=0x1000003b, device_count=1
device[0] core_count=1
config file read network count=1
init test resources, task_count: 1 ...
create/prepare networks ...
task i=0, binary name: ./network_binary.nb
nbg name=./network_binary.nb
create network 0: 1749 us.
input 0 dim 224 224 3 1, data_format=2, quant_format=2, name=input[0], scale=0.003922, zero_point=0
ouput 0 dim 2 1 0 0, data_format=2, name=uid_1_out_0, scale=0.001625, zero_point=128
memory pool size=1092352byte
network core count=1
prepare network 0: 1044 us.
golden file count=0
input 0 name: ./input_0.dat
read input and golden 0: 439 us.
task: 0, loop count: 1
start to run network=./network_binary.nb
run time for this network 0: 3229 us.
run network done...
profile inference time=2876us, cycle=2785921
destroy test resource task_count=1
vpm run ret=0
radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run/etc/npu/vpm_run$This means that we need to use the cc lines to replace build_linux.sh's content but with adjusted file paths based on "awnpu_model_zoo" instead of "ai-sdk" and system OpenCV instead of local OpenCV because both yolo11_demo_linux_a733 and vpm_run use the vip_lite.h API and identical .so shared objects, as investigated here,radxa@radxa-cubie-a7z:~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/common/npuruntime/lib_linux_aarch64/A733$ ls -l total 212 -rwxr-xr-x 1 radxa radxa 174728 Apr 23 07:56 libNBGlinker.so -rwxr-xr-x 1 radxa radxa 39416 Apr 23 07:56 libVIPhal.so radxa@radxa-cubie-a7z:~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/common/npuruntime/lib_linux_aarch64/A733$ ... radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ ls .//usr/include ls: cannot access './/usr/include': No such file or directory radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ ls ../libawnn_viplite awnn_internal.h awnn_lib.c awnn_lib.h awnn_quantize.c awnn_quantize.h radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ ls ../libawutils image_utils.c image_utils.h radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ ls ../../viplite-tina/lib/aarch64-none-linux-gnu/v2.0/inc vip_lite_common.h vip_lite.h radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ ls .//usr/lib ls: cannot access './/usr/lib': No such file or directory radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ ls ../../viplite-tina/lib/aarch64-none-linux-gnu/v2.0/ debug inc libNBGlinker.so libVIPhal.so radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ ls -l ../../viplite-tina/lib/aarch64-none-linux-gnu/v2.0/ total 220 drwxr-xr-x 5 radxa radxa 4096 Jun 27 14:08 debug drwxr-xr-x 2 radxa radxa 4096 Jun 27 14:08 inc -rw-r--r-- 1 radxa radxa 174728 Jun 27 14:08 libNBGlinker.so -rw-r--r-- 1 radxa radxa 39416 Jun 27 14:08 libVIPhal.so radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ find ~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/ -name awnn_quantize.h radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ find ~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/ -name awnn_internal.h radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ find ~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/ -name image_utils.c radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ find ~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/ -name image_utils.h radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ find ~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/ -name vip_lite_common.h /home/radxa/awnpu_model_zoo-v1.0.0-20260423-f562dd16/common/npuruntime/include_85x/vip_lite_common.h /home/radxa/awnpu_model_zoo-v1.0.0-20260423-f562dd16/common/npuruntime/include/vip_lite_common.h radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$ find ~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/ -name vip_lite.h /home/radxa/awnpu_model_zoo-v1.0.0-20260423-f562dd16/common/npuruntime/include_85x/vip_lite.h /home/radxa/awnpu_model_zoo-v1.0.0-20260423-f562dd16/common/npuruntime/include/vip_lite.h radxa@radxa-cubie-a7z:~/ai-sdk/examples/vpm_run$
And the solution is so,
radxa@radxa-cubie-a7z:~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/examples/yolo11$ cat make.sh #radxa@radxa-cubie-a7z:~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/common/npuruntime$ g++ -c npulib.cpp -o npulib.cpp.o -I ../../common/npuruntime/include/ #radxa@radxa-cubie-a7z:~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/common/npuruntime$ g++ -c npu_util.cpp -o npu_util.cpp.o -I ../../common/npuruntime/include/ gcc -I /usr/include/opencv4 -c yolo11_6_pre.cpp -o yolo11_6_pre.cpp.o gcc -I /usr/include/opencv4 -c yolo11_6_post.cpp -o yolo11_6_post.cpp.o gcc -I ../../common/npuruntime/ -c main.cpp -o main.cpp.o g++ main.cpp.o yolo11_6_post.cpp.o yolo11_6_pre.cpp.o ../../common/npuruntime/npulib.cpp.o ../../common/npuruntime/npu_util.cpp.o -o yolo11_demo_a733 \ -L ../../common/npuruntime/lib_linux_aarch64/A733/ \ -lopencv_core -lopencv_imgproc -lopencv_imgcodecs -lNBGlinker -lVIPhal -lm radxa@radxa-cubie-a7z:~/awnpu_model_zoo-v1.0.0-20260423-f562dd16/examples/yolo11$
No comments:
Post a Comment