Fastvideo

Low latency H.264 streaming from Jetson TX2 to PC

Updated:

Author: Fyodor Serzhenko

Remote control of a vision system needs the camera-to-screen path kept as short as possible. This article describes a low-latency H.264 streaming pipeline from NVIDIA Jetson TX2 to a PC, where GPU encoding and a lean transport keep glass-to-glass latency low.

There are a lot of imaging systems which rely on small feedback time, like autonomous cars, hexacopters or drones, mobile robots that should be remotely controlled via wireless network, etc. Quite often these are multicamera systems, because just one camera can't be enough to get full info from controlling objects.

 

multicamera system on jetson

Fig.1. Multicamera system on Jetson TX2

So the task gets more complicated and we will consider a vision system which is based on Jetson TX2 with several cameras, connected directly to Jetson. That scenario is widespread and it can offer remote control over wireless connection.

Glass to Glass Video Latency Test for H264 RTSP Streaming

What glass-to-glass latency can you get for H.264 RTSP streaming from Jetson TX2?

Low delay video streaming is a common task in many applications. Among these applications there are teleoperation scenarios such as controlling drones or telesurgery, virtual or augmented reality, autonomous control of dynamic processes using computer vision algorithms and neural networks which are applied to video in real-time. To evaluate the latency and the performance in such systems, it is important to be able to measure the glass-to-glass (G2G) delay for video acquisition and video transmission.

In general, standard glass to glass video latency test is important to evaluate real performance and latency of imaging system. This is actually the answer to the question about how fast it could be in real life in terms of latency. Glass-to-glass video latency, which is also called end-to-end (E2E) video delay, means the amount of time between the moment when the frame was captured and when it was displayed on the monitor. To be more precise, G2G is describing the time from when the photons of a visible event pass through the lens of the camera till the moment when that image is delivered to the glass of the monitor.

The G2G measurements are usually non-intrusive and they can be applied to a wide range of imaging systems. Furthermore, each video camera has a fixed frame rate, producing new images in constant time intervals. As we know, real world events are never synchronized to the camera frame rate and this is the way to make live real-time measurements, because real world events have to be triggered. That's why obtained G2G delay values are non-deterministic.

 

low latency H.264 streaming on TX2

Fig.2. Low latency video processing

The idea of a simple glass-to-glass test is to show on the monitor two times: the current time and the capture time. This is how we can do that:

Finally, we will see at the same monitor two images: one with the current time and the other with the capture time. The difference between them is the latency of the imaging system.

Here there are several issues at that simple approach that can influence on the final result:

G2G video latency could be evaluated this way, though for some cases such an accuracy could not be sufficient. In these cases more sofisticated hardware could be utilized, but here we consider the current approach to be viable. Due to the above restrictions, we expect to get pretty wide timing interval for video latency.

All our tests will be based on NVIDIA Jetson TX2 System-on-Module

To connect a camera to Jetson TX2, we need a carrier board. Here you can see a carrier board for NVIDIA Jetson TX2 module, specially designed by XIMEA to connect their PCIe and USB3 cameras of xiX / xiC / xiQ series.

xEC2 carrier board interfaces

 

jetson tx2 carrier board

Fig.3. Jetson TX2 carrier board from XIMEA

Finally, up to eight XIMEA cameras can be connected to just one Jetson TX2

The main idea is the following: one could create a multicamera system on Jetson by utilizing that XIMEA carrier board together with up to eight XIMEA cameras and software from MRTech. These are reliable solutions which have already been working in many imaging systems.

Software package

 

Fastvideo SDK TX2

Fig.4. Fastvideo SDK for real-time image and video processing on NVIDIA GPU

Receiving station

RTSP streams from all cameras can be received by using VLC application on one of the devices such as MacBook with Mac OS, desktop or laptop with Windows or Linux OS, Apple devices such as iPhone, iPad, etc.

Full image processing pipeline on Jetson TX2 and Quadro P2000

Benchmarks for one and two-camera systems

These are benchmarks for two XIMEA MX031CG cameras and desktop receiving station with NVIDIA Quadro P2000 GPU over Wi-Fi

Measured glass-to-glass video latency including NVIDIA TX2 H.264 streaming

The above method for G2G evaluation gives us reasonable, though approximate values for the latency of that imaging system. That latency also depends on complexity of video processing pipeline, on image content and on encoding parameters. The above benchmarks for G2G include video processing and video transmission. If you need to compare these results with any other setup, please take that into account.

There are more sofisticated methods for latency measurements which are utilizing light-emitting diode (LED) as a light source and a phototransistor as a light detector. Such a setup for latency measurements includes the blinking LED in the field of view of the camera as signal generator, and tape a photoelectric sensor to where the LED is shown on the display. The LED triggers an oscilloscope which also records the signals from the photoelectric sensor. This allows to extract the G2G delay. The analysis of the data could be done automatically with a microcontroller board. That method could allow to get G2G delay with better precision.

Other blog posts from Fastvideo about Jetson hardware and software

Fyodor Serzhenko, Fastvideo

About the author

Fyodor Serzhenko, PhD, is the founder and CEO of Fastvideo. He earned his PhD at the Moscow Institute of Physics and Technology (MIPT) in 1993. Since 2009 he has led the development of Fastvideo’s GPU-accelerated image codecs and ISP modules, including the Fastvideo SDK. Connect on LinkedIn.

Why you can trust these results

Fastvideo has built GPU-accelerated image processing software since 2009. Our codecs comply with the standards they implement — JPEG (ITU-T T.81 / ISO IEC 10918) and JPEG2000 (ITU-T T.800 / ISO IEC 15444) — and our tools are open source on GitHub. Every performance figure we publish is reproducible: see our benchmark methodology, download the Fastvideo SDK benchmark report (PDF), and measure it on your own GPU.

Contact Form

This form collects your name and email. See our Privacy Policy for how we handle your personal data.
Copyright © fastcompression.com, 2009–2026. All rights reserved. E-mail: info (at) fastcompression.com · Site map