Excitingly Fast GPU Demosaicing Software

Most of modern color cameras don't use 3CCD approach to get color images, because it's complicated and expensive. There is much more simple way to get good colors from image sensors - we need to cover pixels with CFA (color filter array) and after that each pixel will register only R, G or B color component. Due to CFA we decrease the intensity of light and we actually get 1/3 of data, so we need to implement interpolation algorithm to restore all color components at each pixel of the image sensor.

That interpolation algorithm is called demosaicing and it's a must for any photo or video camera with CFA. Demosaicing is a transformation of a Raw Bayer image into the conventional 24/48-bit RGB format. Interpolation of nearby pixel colors is not a simple task and the simplest bayer pattern demosaicing algorithms like Nearest Neighbour or Bilinear give low quality results. To check image quality for demosaicing algorithms, people usually utilize Kodak data set to calculate PSNR or SSIM metrics.

Demosaicing is not only complicated algorithm to restore raw bayer data from image sensors, but also very slow. High quality algorithms like Adaptive Gradient Interpolation or Amaze take several seconds to process raw image from 20-MPix sensor on CPU. Multithreaded software on CPU still can't offer high performance demosaicing for advanced algorithms. Such a performance of video demosaicing software could be acceptable in some cases, but not for realtime processing in camera applications.

To solve the task of fast demosaicing for high quality algorithms, we've implemented highly optimized solutions for NVIDIA GPU. We've improved both quality and the performance significantly. Our raw image demosaicing software can do the following:

• 8/16-bit Raw Bayer image transform to 24/48-bit RGB
• It could be done really fast on NVIDIA GPU, much faster than on any CPU
• Can create images and video with high quality due to sofisticated algorithms
• That software offers significant moire suppression
• Shows immediately time measurements for each demosaic algorithm

What's unique in GPU Demosaicing Software

• This is, probably, the fastest demosaicing software
• Sophisticated algorithms with high PSNR / SSIM and low artifacts
• Simple to use, super fast to work and easy to test
• Free for personal and educational (including non-profit organization) use
• Implemented by a team with years of CUDA programming experience and world-class achievements in image processing

Demosaicing Software Specification

• All Bayer mosaic patterns supported: BGGR, RGGB, GBRG, GRBG
• Mosaic algorithm to produce RAW CFA input image from any 24-bit PPM picture
• Read 8/16-bit input RAW image from HDD/SSD/RAID in PGM format
• Output format: 24/48-bit PPM
• Minimum image resolution 128×128 pixels
• Maximum image resolution up to 16,000×16,000 pixels and more
• Image quality for the following demosaicing algorithms:
  • HQLI demosaic algorithm (High Quality Linear Interpolation, 5×5) – avr. PSNR ~ 36 dB for Kodak data set
  • L7 demosaic algorithm (High Quality Linear Interpolation, window 7×7) – avr. PSNR ~ 37.1 dB (SSIM ~ 0.971) for Kodak data set
  • DFPD demosaic algorithm (Directional Filtering and a Posteriori Decision, 11×11) – avr. PSNR ~ 39 dB for Kodak data set
  • MG demosaic algorithm (Multiple Gradients, 23×23) – avr. PSNR ~ 40.5 dB for Kodak data set
• Maximum demosaic performance rate (without host-to-device and device-to-host transfers) on NVIDIA Quadro P6000 for 8-bit images with 4K resolution:
  • HQLI demosaicing – up to 50 GPix/s
  • L7 demosaicing – up to 45 GPix/s
  • DFPD demosaicing – up to 18 GPix/s
  • MG demosaicing – up to 4 GPix/s
• Software optionally can run on CPU for HQLI, DFPD and MG demosaic algorithms
• Significant color moire artifacts removal
• Quality metrics: MSE, PSNR and SSIM
• Performance tests for each demosaicing algorithm
• Batch mode and tiling for multi-camera systems for real-time video demosaicing - option
• NVIDIA GPU with Compute Capability >= 5.0
• Command-line sample application for Windows-10 and Linux Ubuntu/CentOS and L4T

More info about GPU Demosaic Software

GPU Demosaic is a tool to demonstrate the performance of parallel computations on GPU with CUDA technology. Parallel implementation of GPU demosaicing could be super fast even for very complicated color interpolation algorithms, much faster than realtime demosaicing.

• We have high performance image demosaicing library for your specific needs, particularly for high performance demosaic batch processing
• Here you can see an example of combined demosaic and JPEG compression on NVIDIA GPUs
• We are also offering fast image processing software on NVIDIA GPU for high resolution cameras with the following pipeline: dark frame subtraction, flat field correction, white balance, noise suppression, color correction, tone mapping, rotation, crop, resize, sharp, OpenGL rendering, jpeg/jpeg2000/bayer/h.264/h.265/av1 encoding, etc.
• Camera application Fast VCR with GPU-based image processing
• To check image quality and trade-offs of our demosaicing algorithms, please download Fast CinemaDNG Processor software together with test images in DNG format.

Roadmap for further improvements of GPU Demosaicing Software

• New high quality MG demosaic algorithm (average PSNR for Kodak data set is 40.5 dB) - done
• 16-bit high quality raw image demosaicing for Fast CinemaDNG Processor software - done
• MG demosaic with 16/32-bit precision - done
• Other demosaic optimizations for Volta, Turing, Ampere, Ada