1- How to Generate Engine Using tlt-converter?
2- How to Deploy the Model in Deepstream?
OS: Jetpack 4.5
In this blog post, we will show how to integrate custom segmentation model that we previously trained using NVIDIA® Transfer Learning Toolkit into Jetson™ modules with Deepstream. To learn how to train a custom segmentation model with transfer learning toolkit, click here.
Before we get started, you should download tlt-converter to generate an engine and ds-tlt-segmentation to test the model.
To install tlt-converter go to the following page, download and unzip the zip file for Jetpack 4.5:
You need to install OpenSSL package as well.
sudo apt-get install libssl-dev
Export the following environment variables.
To install ds-tlt, go to following GitHub post and download deepstream-tlt-apps according to the instructions. You do not need to download models since we will use our own model.
Make sure you have exported models that we previously trained.
Now, we will generate an engine using tlt-converter. You can find the usage of tlt-converter by running the following command.
We will show the steps for each exported model (fp16, fp32 and int8).
/home/nvidia/tlt-converter -k forecr -p input_1,1x3x320x320,2x3x320x320,3x3x320x320 segmentation_model_fp16.etlt -e fp16.engine -t fp16
/home/nvidia/tlt-converter -k forecr -p input_1,1x3x320x320,2x3x320x320,3x3x320x320 segmentation_model_fp32.etlt -e fp32.engine -t fp32
Int8 Model (do not forget to add calibration cache file):
/home/nvidia/tlt-converter -k forecr -p input_1,1x3x320x320,2x3x320x320,3x3x320x320 segmentation_model_int8.etlt -e int8.engine -t int8 -c cal.bin
Before running ds-tlt, we need to make changes on the Make files.
Set NVDS_VERSION:=5.1 in apps/Makefile and post_processor/Makefile.
Now, you can set the CUDA version and run Make files.
export CUDA_VER=xy.z // xy.z is CUDA version, e.g. 10.2
Also, go to deepstream source code and change input/output dimensions according to your model. For example, for Resnet18-3 channel model set the following.
#define MODEL_OUTPUT_WIDTH 320
#define MODEL_OUTPUT_HEIGHT 320
Finally, you can run ds-tlt-segmentation to test the model. Supported formats for the test input are jpeg and h264. You can use the sample videos and pictures that comes with deepstream, under opt/nvidia/deepstream/deepstream-5.0/samples/streams.
./apps/tlt_segmentation/ds-tlt-segmentation -c configs/unet_tlt/pgie_unet_tlt_config.txt -i streams/sample_720p.jpeg
Thank you for reading our blog post.