ALTMapper performance benchmarks on NVIDIA GPU
Updated:
These benchmarks measure Fastvideo's adaptive local tone mapping (ALTMapper) on the GPU across 2K, 4K, 8K, and 65 MPix images. On an NVIDIA GeForce RTX 4090 the engine processes a 65 MPix 16-bit frame in 6.3 ms (10 GPix/s); the full results for desktop GPU, Jetson, and CPU are below.
The global tone mapping algorithm is pretty simple. It uses the same curve or function for every single pixel in the image. It does this based on the image's global statistics, like the average luminance. When we do it that way, we lose detail in both the brightest (highlights) and darkest (shadows) parts of the image at the same time. The image looks flat and unnatural. Actually, the local tone mapping algorithm adjusts the brightness pixel by pixel and region by region based on the local context of the surrounding pixels.
Key Advantages of ALTMapper
- Simultaneous Detail Preservation: This is the primary reason. ALTMapper can make the bright areas (like the sky) the same or a little bit darker and the dark areas (like the forest shadows) lighter. This lets you see texture in the clouds and details in the forest in the same image.
- Enhanced Perceived Contrast: ALTMapper can boost local contrast while compressing the global dynamic range. It can make the edges and textures in the middle tones sharper and more defined, which makes the image look more vivid.
- More Natural and Visually Pleasing Results: ALTMapper treats different parts of the image differently, so the result feels more balanced and closer to what we remember seeing with our own eyes. It avoids the flat, "over-processed" look of global operators.
We need local tone mapping because it is an essential computational bridge between the physical reality of light and the limitations of our digital displays. It allows us to faithfully reproduce the rich visual experience of the real world with all its bright highlights and deep shadows on a standard screen, preserving detail, enhancing contrast, and creating a more natural and compelling image.
16-bit TIFF is on the left, ALTMapper result is on the right (source image from the HDR+ dataset)
Here's some info about how fast that algorithm performs on mobile and desktop GPUs from NVIDIA. This will help you to get a handle on its processing speed.
Images for evaluation
- Image file format – PPM, 3 channels, 16-bit per channel
- Standard image resolutions for testing: 1920 × 1080 (2K), 3840 × 2160 (4K), 7680 × 4320 (8K)
- Additional high resolutions: 9344 × 7000 (63 MPix), 11272 × 9200 (99 MPix), 14204 × 10652 (148 MPix)
Hardware and software for NVIDIA Jetson configurations
- Jetson Orin NX 8GB (Ampere, 32 SMMs, 1024 cores) with 8-core ARM, Jetpack 6.0, CUDA Toolkit 12.2
- Jetson Orin AGX 64GB (Ampere, 64 SMMs, 2048 cores) with 12-core ARM, Jetpack 6.0, CUDA Toolkit 12.2
Hardware and software for desktop configuration
- GPU NVIDIA GeForce RTX 4090 (Ada Lovelace, 128 SMMs, 16384 cores) with CPU AMD Ryzen9 7950X (16 cores)
- OS Windows 11 Pro (x64), version 23H2, CUDA Toolkit 12.6
How we've done performance measurements
We've done time and performance measurements for the ALTMapper algorithm for 48-bit color images with 2K, 4K, and 8K resolutions, and more. All the results we've got here don't include any host I/O latency. That's the time it takes to load an image into RAM from a hard disk or solid-state drive, and also to save it back. We also didn't include the time it takes to transfer data between devices. We've made an assumption about how to use the tone mapper in our standard image processing pipeline, when the initial data is stored in GPU memory. Check out the table below for our averaged measurement results for the best series of 1000 processed frames. We've been using 16-bit RGB images for evaluation because that's our standard internal format for image processing on GPU with our SDK on CUDA.
| GPU \ Image resolution | 1920 × 1080 (2K) | 3840 × 2160 (4K) | 7680 × 4320 (8K) | 9344 × 7000 (63 MPix) | 11272 × 9200 (99 MPix) | 14204 × 10652 (148 MPix) |
| Jetson Orin NX 8GB | 24 ms | 44 ms | 72 ms | -- | -- | -- |
| Jetson Orin AGX 64GB | 12 ms | 22 ms | 36 ms | -- | -- | -- |
| GeForce RTX 4090 | 0.35 ms | 1.2 ms | 2.5 ms | 3.8 ms | 6.5 ms | 9.0 ms |
Table 1 - Performance benchmarks for ALTMapper algorithm on Jetson Orin NX/AGX and GeForce RTX 4090
As we can see, the best performance is possible with higher image resolutions. This is actually the case for how well software designed with CUDA is performing when it comes to ISP processing. For a 148 MPix image, we're getting up to 16 GPix/s on the NVIDIA GeForce RTX 4090. The above benchmarks show that we can use ALTMapper for real-time camera applications, particularly with Jetson hardware.
How we've done performance measurements
Our results show that processing time is usually less in comparison with our demosaicing algorithms on GPU. The tonemapper is also faster than our bilateral denoiser on GPU. As soon as tone mapping should be applied close to the end of the ISP pipeline of RAW to RGB conversion, that processing time should not affect much the total performance for such a workflow.
P.S. ALTMapper algorithm has a very small footprint size, so it looks suitable for FPGA implementation, that could be an interesting topic to discuss.
How to verify these numbers: they were measured on the NVIDIA GPU stated above and are reproducible on your own GPU — download the demo apps and measure the speed on your own images.