Image Processing Framework on Jetson

Updated: July 23, 2026

Building a camera application on NVIDIA Jetson means running many processing stages efficiently within a limited embedded power budget. Fastvideo provides a GPU image processing framework for Jetson that runs the full ISP — from RAW acquisition to JPEG and JPEG2000 — entirely on the module's GPU.

From that point of view, NVIDIA Jetson series is suited exactly for the task of high performance image processing from RAW to YUV. Image sensor or camera module can be connected directly to any Jetson via MIPI SCI-2 (2lane/4lane), USB3 or PCIe interfaces. Jetson could offer high performance computations either on ISP or on GPU. Below we show what could be done on GPU. We believe that raw image processing on GPU can offer more flexibility, better performance, quality and ease of management in comparison with hardware-based ISP for many applications.

What is Image Processing Framework for Jetson?

To get high quality and max performance at image processing tasks on Jetson, we've implemented a GPU-based SDK for raw processing. Now we are expanding that approach by creating an effective framework to control all system components, including hardware and software. For example, it means that image sensor control should be included in the workflow at real-time to become a part of general control algorithm.

Image processing framework components

Image Sensor Control (exposure, gain, awb)
Image Capture (driver, hardware/software interface, latency, zero-copy)
RAW Image Processing Pipeline (full raw to rgb workflow)
Image Enhancement
Image/Video Encoding (JPEG/J2K and H.264/H.265)
Compatibility with third-party libraries for image processing, ML, AI, etc.
Image Display Processing (smooth player, OpenGL/CUDA interoperability)
Image/Video Streaming (including interoperability with FFmpeg and GStreamer)
Image/Video Storage

Additional features for the framework

Image Sensor and Lens Calibration
Quality Control for Image/Video Processing
CPU/GPU/SSD balance load, performance optimization, profiling

Implementation of image sensor control at the workflow brings us additional features which are essential. For example, integrated exposure and gain control will allow to get better quality in the case of varying illumination. Apart from that, calibration data usually depend on exposure/gain and it means that we will be able to utilize correct processing parameters at any moment for any viewing conditions.

In general, standard RAW concept eventually lacks internal camera parameters and full calibration data. We could solve that problem by including image sensor control both in calibration and image processing. We can utilize image sensor abstraction layer to take into account full metadata for each frame.

Such a solution depends on utilized image sensor and task to be solved, so we can configure and optimize the Image Processing Framework for a particular image sensor from SONY, Gpixel, CMOSIS image sensors. These solutions on Jetson have already been implemented by teams of Fastvideo and MRTech.

Integrated Image Sensor Control

Exposure time
AWB
Gain
ROI (region of interest)

Full image sensor control also includes bit depth, FPS (frames per second), raw image format, bit packing, mode of operation, etc.

GPU image processing modules on Jetson for 16/32-bit pipeline

Raw image acquisition from image sensor via MIPI/USB3/PCIe interfaces
Frame unpacking
Raw image linearization
Dark frame subtraction
Flat field correction
Dynamic bad pixel removal
White balance
RAW and RGB histograms as an indication to control image sensor exposure time
Demosaicing with L7, DFPD, MG algorithms
Color correction
Denoising with wavelets
Color space and format conversions
Curves and Levels
Flip/Flop, Rotation to 90/180/270 or to arbitrary angle
Crop and Resize (upscale and downscale)
Undistortion via Remap
Local contrast
Tone mapping
Gamma
Real-time output via OpenGL
Trace module for debugging and bug fixing
Stream-per-thread support for better performance
Additional modules: tile support, image split into separate planes, RGB to Gray transform, defringe, etc.
Time measurements for all SDK modules

Image/Video Encoding modules on GPU

RAW Bayer encoding
JPEG encoding (visually lossless image compression with 8-bit or 12-bit per channel)
JPEG2000 encoding (lossy and lossless image compression with 8-16 bits per channel)
H264 encoder/decoder, streaming, integration with FFmpeg (8-bit per channel)
H265 encoder/decoder, streaming, integration with FFmpeg (8/10-bit per channel)

Is it better or faster than NVIDIA ISP for Jetson?

There are a lot of situations where we can say YES to this question. NVIDIA ISP for Jetson is a great product, it's free, versatile, reliable, and it takes less power/load from Jetson, but we have our own advantages which are also of great importance for our customers:

Processing performance
Image quality
Flexibility in building custom image processing pipeline
Wide range of available image processing modules for camera applications
Image processing with 16/32-bit precision
High-performance codecs: JPEG, JPEG2000 (lossless and lossy)
High-performance 12-bit JPEG encoder
Raw Bayer Codec
Dynamic bad pixel suppression
High quality demosaicing algorithms
Wavelet-based denoiser on GPU for Bayer and RGB images
Flexible output with desired image resolution, bit depth, color/grayscale, rotation, according to ML/AI requirements

We've built that software from the scratch and we've been working in that field more than 10 years, so we have an experience and we can offer reliable solutions and support. Apart from that we are offering custom software design to solve almost any problem in a timely manner.

What are benefits of that approach?

That approach allows us to create embedded image processing solutions on Jetson with high quality, exceptional performance, low latency and full image sensor control. Software-based solution in combination with GPU image processing on NVIDIA Jetson could help our customers to create their imaging products with minimum efforts and maximum quality and performance.

About the author

Fyodor Serzhenko, PhD, is the founder and CEO of Fastvideo. He earned his PhD at the Moscow Institute of Physics and Technology (MIPT) in 1993. Since 2009 he has led the development of Fastvideo’s GPU-accelerated image codecs and ISP modules, including the Fastvideo SDK. Connect on LinkedIn.

Why you can trust these results

Fastvideo has built GPU-accelerated image processing software since 2009. Our codecs comply with the standards they implement — JPEG (ITU-T T.81 / ISO IEC 10918) and JPEG2000 (ITU-T T.800 / ISO IEC 15444) — and our tools are open source on GitHub. Every performance figure we publish is reproducible: see our benchmark methodology, download the Fastvideo SDK benchmark report (PDF), and measure it on your own GPU.