de Jeff Tang, Hamid Shojanazeri, Geeta Chauhan
nvidiasupersonic nano, Part ofJetson product familyor Jetson modules, is a small but powerful Linux (Ubuntu) based embedded computer with 2/4 GB GPU. With it, you can run many PyTorch models efficiently. This document summarizes our experience running various deep learning models with three different engines on the Jetson Nano:
Jetson Inference, NVIDIA's high-level API with built-in support for running the most popular computer vision models, which can be ported to the Jetson platform and learned using PyTorch.
TensorRT, a high-performance inference SDK from NVIDIA that requires converting a PyTorch model to ONNX and then to the TensorRT engine file that the TensorRT runtime can execute.
PyTorch with the direct PyTorch API
After purchasing a Jetson NanoHere, just follow the clear step by stepinstructionsto download and write the Jetson Nano Developer Kit SD card image to a microSD card and complete the setup. Once the setup is complete and the Nano boots, you will see the standard Linux prompt along with the username and name of the Nano used during setup.
Run the following commands to check the GPU status on the Nano:
sudo pip3 install jetson-statssudo jtop
You'll see information including:
You can also see the installed CUDA version:
$ ls -lt /usr/locallrwxrwxrwx 1 root root 22. August 2 01:47 cuda -> /etc/alternatives/cudalrwxrwxrwx 1 root root 25. August 01:47 cuda-10 -> /etc/alternatives/cuda-10drwxr- xr - x 12 Return 4096 2 . 01 : 47 } 10.2
To use a camera on the Jetson Nano, for example Arducam 8MP IMX219, follow the instructionsHereor run the following commands afterInstall a camera module:
cd ~wget https://github.com/ArduCAM/MIPI_Camera/releases/download/v0.0.3/install_full.shchmod +x install_full.sh./install_full.sh -m arducam
Another way to do this is to use the original Jetson Nano camera driver:
sudo dpkg -r arducam-nvidia-l4t-kernelsudo off -r now
Then use ls /dev/video0 to confirm that the camera was found:
And finally the following command to see the camera in action:
Using Jetson's inference
nvidiasupersonic lockThe API provides the easiest way to run image recognition, object detection, semantic segmentation, and pose estimation models on Jetson Nano. Jetson Inference has TensorRT built in, so it's very fast.
To test running Jetson Inference, first clone the repository and download the models:
git clone --recursive https://github.com/dusty-nv/jetson-inferencecd jetson-inference
Then use the pre-made onesDocker-ContainerI already installed PyTorch to test the models:
docker/run.sh --volume ~/jetson_inference:/jetson_inference
To run image recognition, object recognition, semantic segmentation, and pose estimation models on test images, use:
cd build/aarch64/bin./imagenet.py images/jellyfish.jpg /jetson_inference/jellyfish.jpg./segnet.py images/dog.jpg /jetson_inference/dog.jpeg./detectnet.py images/peds_0.jpg /jetson_inference /peds_0.jpg./posenet.py images/humans_0.jpg /jetson_inference/pose_humans_0.jpg
Four images of the results of running the four different models are generated. Exit the docker image to view them:
$ls -lt~/jetson_inference/-rw-r--r-- 1 root root 68834 Oct 15 9:30 PM pose_humans_0.jpg-rw-r--r-- 1 root root 914058 Oct 15 9:30 PM peds_0 .jpg -rw-r--r-- 1 root root 666239 October 15 9:30 PM dog.jpeg-rw-r--r-- 1 root root 179760 October 15 9:29 PM jellyfish.jpg
You can also use the Docker image to run PyTorch templates, as the image has PyTorch, Torchvision, and Torchaudio installed:
# lista de pip|grep Torchtorch (1.9.0)torchaudio (0.9.0a0+33b2469)torchvision (0.10.0a0+300a8a4)
Although Jetson Inference includes models already converted to the TensorRT engine file format, you can optimize the models by following the steps in Transfer Learning with PyTorch (for Jetson Inference).Here.
TensorRTis a high-performance inference SDK from NVIDIA. The Jetson Nano supports TensorRT through the Jetpack SDK, which is included in the SD card image used to configure the Jetson Nano. To confirm that TensorRT is already installed on the Nano,
Execute dpkg -l|grep -i tensort:
In theory, TensorRT could be used to "take a trained PyTorch model and tune it to run more efficiently during inference on an NVIDIA GPU". Follow the instructions and code onPortable computerto see PyTorch being used with TensorRT over ONNX in a Torchvision Resnet50 model:
How to convert PyTorch model for ONNX;
How to convert ONNX model to a TensorRT engine file;
To run the engine file with the TensorRT runtime to improve performance: Increased the inference time from the original 31.5ms/19.4ms (FP32/FP16 precision) to 6.28ms (TensorRT) enhanced.
You can replace the Resnet50 model in the notebook code with another PyTorch model, follow the above conversion process, and run the converted final model TensorRT engine file with the TensorRT runtime to see optimized performance. However, please note that due to the memory size of the Nano GPU, models larger than 100MB may not work with the following error information:
Error code 1: Cuda runtime (all CUDA enabled devices are busy or unavailable)
You may also get an error converting a PyTorch model to an ONNX model, which can be fixed by replacing:
Torch.onnx.export(resnet50, dummy_input, "resnet50_pytorch.onnx", detallado=Falso)
Torch.onnx.export(modelo, dummy_input, "deeplabv3_pytorch.onnx", opset_version=11, detallado=Falso)
To download and install PyTorch 1.9 on Nano, first run the following commands (seeHerefor more information):
wget https://nvidia.box.com/shared/static/p57jwntv436lfrd78inwl7iml6p13fzh.whl -O Torch-1.8.0-cp36-cp36m-linux_aarch64.whl -O Torch-1.9.0-cp36-cp36m-linux_aarch64.whlsudo apt- conseguir instalar python3-pip libopenblas-base libopenmpi-dev pip3 instalar Cythonpip3 instalar numpy Torch-1.9.0-cp36-cp36m-linux_aarch64.whl
To download Torchvision 0.10 and install it on the Nano, run the following commands:
https://drive.google.com/uc?id=1tU6YlPjrP605j4z8PMnqwCSoP6sSC91Zpip3 Instala Torchvision-0.10.0a0+300a8a4-cp36-cp36m-linux_aarch64.whl
After above steps, run this to confirm:
$ pip3 list|grep TorchTorch (1.9.0) TorchVision (0.10.0)
You can also use the Docker image described in the sectionUsing Jetson's inference(which also has PyTorch and Torchvision installed) to skip the manual steps above.
The OfficialYOLOv5repo is used to run the PyTorch YOLOv5 model on the Jetson Nano. After logging in to the Jetson Nano, do the following:
- Get the repository and install whatever is needed:
git-Klon https://github.com/ultralytics/yolov5cd yolov5pip install -r requirements.txt
python3 detectar.py, which uses the yolov5s.pt PyTorch template by default. You should see something like:
detectar: weights=yolov5s.pt, source=data/images, imgsz=[640, 640], conf_thres=0.25, iou_thres=0.45, max_det=1000, device=, view_img=False, save_txt=False, save_conf=False , save_crop =False, nosave=False, Classes=None, agnostic_nms=False, Augment=False, visualize=False, update=False, Project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False , hide_conf =False, Medium=FalseYOLOv5 🚀 v5.0-499-g48b00db Torch 1.9.0 CUDA:0 (NVIDIA Tegra X1, 3956.1015625MB) Fusion Layers... Model Summary: 224 layers, 7266973 parameters, 0 gradientsImage 1/5 /home /jeff /repos/yolov5-new/yolov5/data/images/bus.jpg: 640x480 4 pessoas, 1 ônibus, 1 hidrante, pronto. (0,142s)...
The inference time on the Jetson Nano GPU is approximately 140ms, more than double the inference time on iOS or Android (approximately 330ms).
If you receive an error message
"ImportError: _imagingft C module is not installed."then you need to reinstall the pillow:
sudo apt-get install libpng-devsudo apt-get install libfreetype6-devpip3 desinstalar Pillowpip3 install --no-cache-dir Travesseiro
After successfully completing the
python3 detectar.pyrunning, the object detection results of the test patterns are in
data/imageswill be in
run/acknowledge/expDirectory. To test detection using a live webcam instead of local images, use the
--source 0parameters at runtime
~/repos/yolov5$ ls -lt running/detect/exp10total 1456-rw-rw-r-- 1 jeff jeff 254895 15. Out 4:12 pm zidane.jpg-rw-rw-r-- 1 jeff jeff 202674 15 . Oct 16:12 test3.png-rw-rw-r-- 1 jeff jeff 217117 15. Oct 16:12 test2.jpg-rw-rw-r-- 1 jeff jeff 305826 15. Oct 16:12 test1 .png- rw-rw - r-- 1 jeff jeff 495760 october 15 4h12 bus.jpg
Using the same test files used in PyTorch iOS YOLOv5 Demo App or Android YOLOv5 Demo App, you can compare the results obtained by running the YOLOv5 PyTorch model on mobile devices and Jetson Nano were generated:
Figure 1. PyTorch YOLOv5 on Jetson Nano.
Figura 2. PyTorch YOLOv5 no iOS.
Figura 3. PyTorch YOLOv5 no Android.
Based on our experience running various PyTorch models for potential demo applications on the Jetson Nano, we see that even the Jetson Nano, a lower end of the Jetson family of products, offers a powerful GPU and an integrated system that can run some of the most demanding models. recent versions of PyTorch directly, pre-trained or transferred, efficient.
Building PyTorch demo apps on Jetson Nano can be similar to building PyTorch apps on Linux, but you can also use TensorRT after converting your PyTorch models to the TensorRT engine file format.
But if you just need to run some common computer vision models on the Jetson Nano using NVIDIA's Jetson Inference, which supports image recognition, object recognition, semantic segmentation, and pose estimation models, this is the easiest way.
Torch-TensorRT, a compiler for PyTorch via TensorRT:https://github.com/NVIDIA/Torch-TensorRT/
A guide to using TensorRT on NVIDIA Jetson Nano:https://docs.donkeycar.com/guide/robot_sbc/tensorrt_jetson_nano/Including:
Use Jetson as a portable GPU device to run an NN chess engine model:https://medium.com/@ezchess/jetson-lc0-running-leela-chess-zero-on-nvidia-jetson-a-portable-gpu-device-a213afc9c018
A MaskEraser application using PyTorch and Torchvision, installed directly with pip :https://github.com/INTEC-ATI/MaskEraser#install-pytorch