How to Run Yolov5 Real Time Object Detection on NVIDIA® Jetson™ Nano™?

How to Run Yolov5 Real Time Object Detection on NVIDIA® Jetson™ Nano™?

Jetson Nano

12 August 2021
WHAT YOU WILL LEARN?

1- How to use Yolov5 model files in docker.

2- How to use Yolov5 Object Detection with both USB webcam and CSI camera.

3- How to run Yolov5 Object Detection in docker.

ENVIRONMENT

Hardware: DSBOX-N2

OS: Ubuntu 18.04 LTS, JetPack 4.5


How to use Yolov5 model files in docker


 In this blog post, you will learn how to run Yolov5 Object Detection in real time with both a USB camera, and a CSI camera. The GitHub repo has been taken as a reference for the whole process.


 If you are going to use a CSI camera for object detection, you should connect it to Jetson™ Nano™ before powering it up.

First, let us go to the Documents folder, then let us install the required files while in this folder:


cd ./Documents 
git clone https://github.com/amirhosseinh77/JetsonYolo.git


Now, let us go into the weights folder and download assets for Yolov5:


cd weights 
wget https://github.com/ultralytics/yolov5/releases/download/v5.0/yolov5s.pt


And the final requirement for our setup, installing Gstreamer, let us install it:

sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio 
sudo apt-get install gstreamer1.0-tools

How to use Yolov5 Object Detection with both USB webcam and CSI camera


 Since our preparations outside the docker are done, let us install our container and run it:

cd 
docker pull nvcr.io/nvidia/l4t-ml:r32.5.0-py3
sudo docker run -it --rm --gpus all -e DISPLAY=:0 -v /tmp/.X11-unix:/tmp/.X11-unix -v /home/nvidia/Documents:/home/nvidia/Documents -v /tmp/argus_socket:/tmp/argus_socket --device /dev/video0:/dev/video0 --network host nvcr.io/nvidia/l4t-ml:r32.5.0-py3


Now, we should be inside our container. We need to create, write and run a python file, and in order to do that, we need to install nano, Tkinter and tqdm:

apt-get update 
apt-get install nano
apt-get install python3-tk
pip3 install tqdm


After we got these installations, we need to create a python file inside JetsonYolo folder, and write our code:

cd /home/nvidia/Documents/JetsonYolo/ 
touch JetsonYolo.py
nano JetsonYolo.py


Now you should see an empty file like below:



Now, if you are going to use a USB Webcam, copy the code below and paste it with Ctrl + Shift + P:

import cv2 
import numpy as np
from elements.yolo import OBJ_DETECTION

Object_classes = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',                'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',                'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',                'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',                'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',                'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',                'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone',                'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear',                'hair drier', 'toothbrush' ]

Object_colors = list(np.random.rand(80,3)*255)
Object_detector = OBJ_DETECTION('weights/yolov5s.pt', Object_classes)

cap = cv2.VideoCapture(0)
if cap.isOpened():
     window_handle = cv2.namedWindow("USB Camera", cv2.WINDOW_AUTOSIZE)
     # Window
     while cv2.getWindowProperty("USB Camera", 0) >= 0:
         ret, frame = cap.read()
         if ret:
             # detection process
             objs = Object_detector.detect(frame)

             # plotting
             for obj in objs:
                 # print(obj)
                 label = obj['label']
                 score = obj['score']
                 [(xmin,ymin),(xmax,ymax)] = obj['bbox']
                 color = Object_colors[Object_classes.index(label)]
                 frame = cv2.rectangle(frame, (xmin,ymin), (xmax,ymax), color, 2)
                 frame = cv2.putText(frame, f'{label} ({str(score)})', (xmin,ymin),
                 cv2.FONT_HERSHEY_SIMPLEX , 0.75, color, 1, cv2.LINE_AA)

         cv2.imshow("USB Camera", frame)
         keyCode = cv2.waitKey(30)
         if keyCode == ord('q'):
             break
     cap.release()
     cv2.destroyAllWindows()
else:
     print("Unable to open camera")

If you are going to use a CSI camera, then copy below code and paste it with Ctrl + Shift + P:import cv2
import numpy as np
from elements.yolo import OBJ_DETECTION

Object_classes = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',                'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',                'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',                'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',                'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',                'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',                'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone',                'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear',                'hair drier', 'toothbrush' ]

Object_colors = list(np.random.rand(80,3)*255)
Object_detector = OBJ_DETECTION('weights/yolov5s.pt', Object_classes)

def gstreamer_pipeline(
     capture_width=1280,
     capture_height=720,
     display_width=800,
     display_height=600,
     framerate=30,
     flip_method=2,
):
     return (
         "nvarguscamerasrc ! "
         "video/x-raw(memory:NVMM), "
         "width=(int)%d, height=(int)%d, "
         "format=(string)NV12, framerate=(fraction)%d/1 ! "
         "nvvidconv flip-method=%d ! "
         "video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
         "videoconvert ! "
         "video/x-raw, format=(string)BGR ! appsink"
         % (
             capture_width,
             capture_height,
             framerate,
             flip_method,
             display_width,
             display_height,
         )
     )
# To flip the image, modify the flip_method parameter (0 and 2 are the most common)
print(gstreamer_pipeline(flip_method=2))
cap = cv2.VideoCapture(gstreamer_pipeline(flip_method=2), cv2.CAP_GSTREAMER)
if cap.isOpened():
     window_handle = cv2.namedWindow("CSI Camera", cv2.WINDOW_AUTOSIZE)
     # Window
     while cv2.getWindowProperty("CSI Camera", 0) >= 0:
         ret, frame = cap.read()
         if ret:
             # detection process
             objs = Object_detector.detect(frame)

             # plotting
             for obj in objs:
                 # print(obj)
                 label = obj['label']
                 score = obj['score']
                 [(xmin,ymin),(xmax,ymax)] = obj['bbox']
                 color = Object_colors[Object_classes.index(label)]
                 frame = cv2.rectangle(frame, (xmin,ymin), (xmax,ymax), color, 2)
                 frame = cv2.putText(frame, f'{label} ({str(score)})', (xmin,ymin),
                 cv2.FONT_HERSHEY_SIMPLEX , 0.75, color, 1, cv2.LINE_AA)

         cv2.imshow("CSI Camera", frame)
         keyCode = cv2.waitKey(30)
         if keyCode == ord('q'):
             break
     cap.release()
     cv2.destroyAllWindows()
else:
     print("Unable to open camera")

How to run Yolov5 Object Detection in docker


Now, we need to gain access to our camera from docker. Open a new terminal using Ctrl + Alt + T, and write the following:

xhost + 


We should see the following output from the terminal. Then let’s switch back to our first terminal and run our code, since we are ready:

python3 JetsonYolo.py 


Running our file may take a couple of minutes. You can see how it works below:



Press ‘q’ to stop the program.

Thank you for reading our blog post. 


English