ISP acceleration and performance optimization on CPU and GPU
Updated:
Fastvideo offers ISP acceleration services: we profile an existing camera image pipeline and move its heavy stages to the GPU, cutting per-frame time and raising throughput on CPU and NVIDIA GPU. The result is a faster ISP without changing the camera or the image quality you rely on.
What ISP acceleration covers
ISP acceleration means keeping the image quality of your current pipeline while making it run faster. We profile the existing ISP, find the stages that dominate per-frame time, and move them to the GPU, so the same processing fits into a smaller time budget and a higher frame rate.
Stages that usually dominate the time budget
- Demosaicing (debayer) and color conversion
- Noise reduction and sharpening
- Resize, remap, undistortion and rotation
- Tone mapping and 3D LUT color grading
- JPEG, JPEG 2000 or H.264/H.265 encoding
How we accelerate a pipeline
- Move the heavy stages to the GPU with our Image & Video Processing SDK
- Overlap data transfer and computation with CUDA streams
- Fuse pipeline stages to cut intermediate memory traffic
- Use zero-copy and unified memory on NVIDIA Jetson
- Batch frames where latency allows, to raise throughput
Latency and throughput are different goals
Throughput is how many frames per second the pipeline sustains; latency is how long a single frame takes from capture to result. Real-time capture needs enough throughput to keep up with the sensor, while glass-to-glass systems also need low latency. We measure both and tune for the one that matters to your application.
What we need to scope the work
- Your current pipeline and where the bottleneck is
- Target frame rate and, if relevant, latency
- Hardware: CPU, NVIDIA GPU or Jetson model, camera interface
- Image sensor, resolution, bit depth and pixel format
The general engagement process is described on our ISP tuning page; for measured performance figures see our GPU benchmarks.
About the author
Fyodor Serzhenko, PhD, is the founder and CEO of Fastvideo. He earned his PhD at the Moscow Institute of Physics and Technology (MIPT) in 1993. Since 2009 he has led the development of Fastvideo’s GPU-accelerated image codecs and ISP modules, including the Fastvideo SDK. Connect on LinkedIn.
Why you can trust these results
Fastvideo has built GPU-accelerated image processing software since 2009. Our codecs comply with the standards they implement — JPEG (ITU-T T.81 / ISO IEC 10918) and JPEG2000 (ITU-T T.800 / ISO IEC 15444) — and our tools are open source on GitHub. Every performance figure we publish is reproducible: see our benchmark methodology, download the Fastvideo SDK benchmark report (PDF), and measure it on your own GPU.