GPU Demosaicing Algorithms and Image Quality
Updated:
Fastvideo GPU demosaicing converts Bayer CFA data to RGB on NVIDIA GPUs with a choice of quality levels — HQLI, L7, DFPD, and MG — reaching from about 36 dB (HQLI) to about 40.5 dB (MG) average PSNR on the Kodak data set. All of them run in real time on the GPU, so you can trade quality against speed without leaving the pipeline.
Looking for the software? Use the GPU Debayer application to convert Bayer to RGB, or add the CUDA Debayer Library to your own pipeline. This page explains the demosaicing algorithms behind them.
Detailed benchmark data. Measured results are published on our benchmark pages: Debayer benchmarks, all benchmarks.
That interpolation algorithm is called demosaicing and it's a must for any photo or video camera with CFA. Demosaicing is a transformation of a Raw Bayer image into the conventional 24/48-bit RGB format. Interpolation of nearby pixel colors is not a simple task and the simplest bayer pattern demosaicing algorithms like Nearest Neighbour or Bilinear give low quality results. To check image quality for demosaicing algorithms, people usually utilize Kodak data set to calculate PSNR or SSIM metrics.
Demosaicing is not only complicated algorithm to restore raw bayer data from image sensors, but also very slow. High quality algorithms like Adaptive Gradient Interpolation or Amaze take several seconds to process raw image from 20-MPix sensor on CPU. Multithreaded software on CPU still can't offer high performance demosaicing for advanced algorithms. Such a performance of video demosaicing software could be acceptable in some cases, but not for real-time processing in camera applications.
To solve the task of fast demosaicing for high quality algorithms, we've implemented highly optimized solutions for NVIDIA GPU. We've improved both quality and the performance significantly. Our raw image demosaicing software can do the following:
8/16-bit Raw Bayer image transform to 24/48-bit RGB- It could be done really fast on NVIDIA GPU, much faster than on any CPU
- Can create images and video with high quality due to sophisticated algorithms
- Shows immediately time measurements for each demosaic algorithm
What's unique in GPU Demosaicing Software
- This is, probably, the fastest demosaicing software
- Sophisticated algorithms with high PSNR / SSIM and low artifacts
- Simple to use, super fast to work and easy to test
- Free for personal and educational (including non-profit organization) use
- Implemented by a team with years of CUDA programming experience and world-class achievements in image processing
|
Fast GPU Demosaicing Software Size: 50.0 MB |
Demosaicing Software Specification
Which demosaicing algorithms and speeds does the GPU software offer?
- All Bayer mosaic patterns supported: BGGR, RGGB, GBRG, GRBG
- Mosaic algorithm to produce RAW CFA input image from any 24-bit PPM picture
- Read 8/16-bit input RAW image from HDD/SSD/RAID in PGM format
- Output format: 24/48-bit PPM
- Minimum image resolution 128×128 pixels
- Maximum image resolution up to 16,000×16,000 pixels and more
- Image quality for the following demosaicing algorithms:
- HQLI demosaic algorithm (High Quality Linear Interpolation, 5×5) – avr. PSNR ~ 36 dB for Kodak data set
- L7 demosaic algorithm (High Quality Linear Interpolation, window 7×7) – avr. PSNR ~ 37.1 dB (SSIM ~ 0.971) for Kodak data set
- DFPD demosaic algorithm (Directional Filtering and a Posteriori Decision, 11×11) – avr. PSNR ~ 39 dB for Kodak data set
- MG demosaic algorithm (Multiple Gradients, 23×23) – avr. PSNR ~ 40.5 dB for Kodak data set
- Maximum demosaic performance rate (without host-to-device and device-to-host transfers, no batch, no streaming) on NVIDIA GeForce RTX 4090 for 16-bit images with 65 MPix resolution:
- HQLI demosaicing – up to 69 GPix/s
- L7 demosaicing – up to 62 GPix/s
- DFPD demosaicing – up to 48 GPix/s
- MG demosaicing – up to 20 GPix/s
- Software optionally can run on CPU for HQLI, DFPD and MG demosaic algorithms
- Significant color moire artifacts removal
- Quality metrics: MSE, PSNR and SSIM
- Performance tests for each demosaicing algorithm
- Command-line sample application for Windows-10/11, Linux, L4T
More info about GPU Demosaic Software
GPU Demosaic is a tool to demonstrate the performance of parallel computations on GPU with CUDA technology. Parallel implementation of GPU demosaicing could be super fast even for very complicated color interpolation algorithms, much faster than real-time demosaicing.
- We have high performance image demosaicing library for your specific needs, particularly for high performance demosaic batch processing
- Here you can see an example of combined demosaic and JPEG compression on NVIDIA GPUs
- We are also offering fast image processing software on NVIDIA GPU for high resolution cameras with the following pipeline: dark frame subtraction, flat field correction, white balance, noise suppression, color correction, tone mapping, rotation, crop, resize, sharp, OpenGL rendering, jpeg/jpeg2000/bayer/h.264/h.265/av1 encoding, etc.
- Camera application Fast VCR with GPU-based image processing
- To check image quality and trade-offs of our demosaicing algorithms, please download Fast CinemaDNG Processor software together with test images in DNG format.
GPU Demosaicing FAQ
What is demosaicking?
Demosaicking (also spelled demosaicing) is the reconstruction of a full-color RGB image from the raw single-channel data of a color-filter-array (CFA) sensor, most commonly a Bayer pattern. It is the same operation that camera applications call debayering.
What is CFA interpolation?
CFA interpolation (color filter array interpolation) is another name for demosaicing: each pixel of a Bayer sensor records only one color, and the two missing colors are interpolated from neighboring pixels to produce a full RGB image.
How is demosaicing quality measured?
Quality is measured as average PSNR against a reference image (here on the Kodak data set), together with visual inspection for zipper artifacts and false color along edges. Our algorithms range from about 36 dB (HQLI) to about 40.5 dB (MG).
Why do gradient-based algorithms beat simple bilinear demosaicing?
Bilinear interpolation ignores edges, so it produces zipper artifacts and false color. Gradient-based methods such as DFPD and MG interpolate along edges rather than across them, which sharply reduces those artifacts at the cost of more computation.
Roadmap for further improvements of GPU Demosaicing Software
- New high quality MG demosaic algorithm (average PSNR for Kodak data set is 40.5 dB) - done
- 16-bit high quality raw image demosaicing for Fast CinemaDNG Processor software - done
- MG demosaic with 16/32-bit precision - done
- New high performance demosaicing algorithm MG2 with better image quality - in progress
About the author
Fyodor Serzhenko, PhD, is the founder and CEO of Fastvideo. He earned his PhD at the Moscow Institute of Physics and Technology (MIPT) in 1993. Since 2009 he has led the development of Fastvideo’s GPU-accelerated image codecs and ISP modules, including the Fastvideo SDK. Connect on LinkedIn.
Why you can trust these results
Fastvideo has built GPU-accelerated image processing software since 2009. Our codecs comply with the standards they implement — JPEG (ITU-T T.81 / ISO IEC 10918) and JPEG2000 (ITU-T T.800 / ISO IEC 15444) — and our tools are open source on GitHub. Every performance figure we publish is reproducible: see our benchmark methodology, download the Fastvideo SDK benchmark report (PDF), and measure it on your own GPU.