Fast RAW Image Encoding on NVIDIA GPU

Performance of image processing in camera applications could be much higher if we apply image compression for raw frames without using debayer. That approach is usually called Raw Bayer Compression. Right after image acquisition we can split RAW image to four planes according to colors of Bayer pattern (RGGB). Then we can do JPEG image compression for each plane with high quality and store compressed image to SSD. It could be done very fast on GPU and without introducing significant image artifacts. In this way we can temporary exclude debayer from image processing pipeline and increase performance for realtime applications.

This is the main idea - to store RAW image with visually lossless encoding. It is the way to preserve original raw frames for further white balance, exposure correction, debayering and color grading at post-production.

In the task of realtime data acquisition one should set JPEG quality at least 90% or more. This is so called set of parameters for visually lossless compression. JPEG quality 100% or lossy JPEG2000 are preferrable, but in that case compression ratio and performance could be not high enough. Optimal compression quality is very important, because strong JPEG image compression can introduce additional image artifacts, which will lead to image artifacts at RAW-to-RGB conversion and at post processing.


bayer image encoding


Pipeline for RAW image encoding on the GPU

  • Image acquisition and unpacking of RAW data
  • Optional black/white point plus 1D LUT
  • Decomposition (split) and alignment of bayer pattern to 4 separate color planes
  • Creating a new image with width/4 and height*4
  • 8-bit or 12-bit JPEG encoding (quality 95-98%)
  • Data storage to SSD/NVMe

After such image acquisition we finally get compressed RAW frames at SSD. Actually we are encoding 1-channel image which leads to good compression ratio and fast performance. Then we need to solve the task of image processing and visualizing for compressed RAW images.

GPU pipeline for RAW decoding, processing and visualization

  • Image decompression (JPEG decoding)
  • Building an original uncompressed RAW image with bayer pattern
  • RAW image linearization
  • White balance
  • Debayer
  • Denoiser
  • Color correction
  • LUT (1D, 3D)
  • Crop/Rotate/Resize/Sharp
  • Gamma
  • OpenGL output
  • Optional output MJPEG compression to AVI or image encoding to JPEG

The above pipeline is computationally demanding and it's not easy to achieve high performance for that task. For visualizing we need to reach up to 60 fps, and for most resolutions this could be done on NVIDIA GPU in realtime. We utilize our image processing SDK for the above pipeline and it can handle data rates up to 4 GPix/s on GeForce RTX 4090 for the case with built-in JPEG compression on NVIDIA GPU. Such a solution is available for all NVIDIA GPUs: mobile, laptop, desktop, server.

We can supply the software for each stage of the task

  • Camera application for RAW image acquisition and image sensor control
  • RAW image compression on Jetson or on any other NVIDIA GPU, storage to SSD
  • RAW to JPEG conversion on NVIDIA GPU

To improve image quality for that application, we could run camera calibration procedures, including FFC, color/tona calibration, undistortion, etc.

Optional methods for 12-bit RAW encoding on the GPU

  • 12-bit RAW image conversion to 8-bit, split, encoding with JPEG codec
  • 12-bit RAW image split, encoding with 12-bit JPEG codec
  • 12-bit RAW image split, encoding with JPEG2000 codec (lossy or lossless)

We also have more sofisticated methods for RAW image compression which are applicable for 14-bit and 16-bit RAW frames from photo and video cameras. These methods also take into account camera noise to offer high image quality at the restoration stage.

Performance benchmarks for 12-bit RAW encoding on the GPU

  • 12-bit 4K RAW image compression with 12-bit JPEG codec on NVIDIA Jetson Orin - 2 ms per frame (q = 95, psnr = 66 dB)
  • 12-bit 4K RAW image compression with 12-bit JPEG codec on NVIDIA GeForce RTX 4090 - 0.35 ms per frame (q = 95, psnr = 66 dB)

Useful links on the subject

Contact Form

This form collects your name and email. Check out our Privacy Policy on how we protect and manage your personal data.