FastVCR Software for MIPI CSI2 Cameras: Real-time ISP on NVIDIA Jetson

Updated: July 29, 2026

FastVCR for MIPI CSI2 cameras runs the complete image signal processor on the GPU of NVIDIA Jetson. Instead of relying on the fixed-function hardware ISP of the Jetson module, it takes RAW frames straight from the sensor and executes every processing stage — unpacking, linearization, demosaicing, color correction, denoising, tone transforms, and encoding — in CUDA, on the same module the camera is attached to. The result is a fully tunable, embedded camera pipeline with real-time display, JPEG and video recording, and low-latency RTSP streaming.

What Is MIPI CSI2

MIPI CSI2 (Camera Serial Interface 2) is the dominant standard for connecting image sensors directly to an application processor. Born in mobile devices, it is now the default camera interface in embedded systems: it moves sensor data at high speed over a scalable number of lanes, consumes little power, and takes almost no board space, which is why multi-camera CSI2 setups are common in robotics, drones, automotive systems, and compact medical devices. NVIDIA Jetson modules expose CSI2 ports natively, making the Jetson + MIPI camera combination the standard building block of embedded vision.

Hardware ISP vs GPU ISP on Jetson

Can you replace the Jetson hardware ISP and libargus with a GPU ISP?

The conventional way to process a MIPI camera on Jetson is the built-in hardware ISP accessed through the libargus library. It is power-efficient, but it is also a black box: the set of processing stages is fixed, the tuning options are limited, and the image quality ceiling is defined by the silicon, not by your requirements. FastVCR takes the other route — a software ISP implemented in CUDA on the Jetson GPU. Every stage of the pipeline is open to tuning, stages can be added or reordered for the task at hand, and quality-critical steps such as demosaicing and denoising use our high quality algorithms rather than fixed-function approximations.

One requirement applies: FastVCR needs a camera driver that exposes RAW frames and the sensor's modes of operation. Most industrial MIPI camera vendors provide such drivers for Jetson.

Supported MIPI CSI2 Cameras

FastVCR runs on a MIPI CSI2 camera whose Jetson driver exposes RAW frames and the sensor’s modes of operation:

• Sony IMX477 (12 MPix) — the reference sensor for our Jetson Orin benchmarks
• Any MIPI CSI2 image sensor with a Jetson RAW driver from the camera vendor — monochrome or Bayer, 8/10/12/14/16-bit

Tell us your sensor and camera vendor in the contact form below, and we will confirm support.

Camera Control Software

• Camera initialization: camera model, default values for frame resolution, fps, etc.
• Input bit depth and pixel format
• Image resolution and region of interest (ROI)
• Frame rate (fps) and/or exposure time
• Gain
• Automatic exposure/gain control

16-bit ISP Pipeline on the Jetson GPU

• Image acquisition and frame unpacking
• Image linearization
• Dark frame subtraction (FPN)
• Flat-field correction (shading correction)
• Bad pixel removal
• White balance / AWB
• Adaptive exposure and gain control
• High quality demosaicing with the MG algorithm
• Color correction with matrix profile or DCP profile
• Highlight recovery
• Exposure correction (brightness control)
• Curves and levels
• Denoising: NLM and bilateral
• Rotation to 90/180/270 degrees and flip/flop; rotation to an arbitrary angle
• Crop, resize (downscale and upscale)
• Undistortion via LCP or via calibrated maps
• Sharpening (local contrast)
• Gamma transform

Recording, Direct-to-Disk Capture, Streaming and Output

• Real-time video output to monitor via OpenGL
• JPEG compression with image storage on SSD
• Optional conversion to NV12 with video encoding to MJPEG (AVI) and H.264/H.265/AV1 (MP4), stored to a video container on SSD
• Automatic real-time partitioning of AVI/MP4 files to a specified file size
• Built-in RTSP server for low-latency streaming; RTSP player included, and the stream is compatible with VLC
• Real-time RAW/PGM data storage on SSD
• Interoperability with third-party GPU-based SDKs and AI libraries at the GPU level

Monitoring

• Histograms and RGB parade
• Camera statistics
• Time measurements for every image processing module on GPU

Performance on Jetson

How fast is MIPI camera processing on Jetson Orin?

The reference configuration: a 12-MPix MIPI camera with the Sony IMX477 image sensor (4056×3040, 12-bit) processed in real time at 30 fps on the NVIDIA Jetson Orin. The measured pipeline covers RAW import, RAW-to-RGB conversion, color correction, JPEG encoding, and storage to SSD; GPU processing takes about 20 ms per frame on this hardware. Actual performance depends on the Jetson model, the pipeline configuration, and the storage; the built-in per-module timing makes it easy to profile your own setup.

Latency

On Jetson, latency is measured with the built-in glass-to-glass (G2G) module: the CSI2 camera captures a timer shown on the display, the frame passes through the CUDA pipeline, and it appears beside the timer — their difference is the end-to-end delay. A codec test extends this through MJPEG or H.264/H.265 encode and decode on a single Jetson or across a link. Since the number depends on sensor fps, display refresh, Jetson model, and pipeline depth, treat it as an estimate and measure it on your target module, as we do for the FastVCR for XIMEA build.

FastVCR vs the Jetson Hardware ISP: What the GPU Adds

Against the Jetson’s fixed-function hardware ISP and libargus, FastVCR trades a black box for a pipeline you own — running in CUDA on the same module the CSI2 camera is attached to.

Open, reorderable stages. Every ISP step is exposed and can be tuned, added, or reordered; demosaicing and denoising use high quality algorithms instead of silicon approximations fixed by the SoC.

On-module recording that keeps up. GPU MJPEG on the same Jetson records resolution and frame-rate combinations past hardware-encoder limits, straight to SSD.

Low, measurable embedded latency. RTSP plus a built-in G2G test quantify glass-to-glass delay on the exact Jetson you deploy, not a reference board.

Quality denoising on the edge. Bilateral and NLM denoisers for monochrome and color, within the module’s power envelope.

Develop before the sensor is wired. RAW/PGM/TIFF sequences from SSD stand in for the live CSI2 camera during bring-up and regression testing.

Headless-first. A remotely controllable CLI suits fanless, display-less Jetson deployments in robotics, drones, and automotive; the GUI is there when you need it.

Per-module timing. Histogram, RGB parade, and stage-by-stage GPU timing for profiling on constrained hardware.

Optional Image Processing Modules on GPU

• RAW frames capture, preview, and storage to SSD
• Histograms for RAW and processed data
• Curves and levels for RAW, RGB, HSV
• DCP and LCP support
• 3D LUT support
• Geometric and affine transforms
• HDR support
• Local and global tone mapping
• Chromatic aberration removal in the RAW domain — in progress
• Multiple camera support
• Camera and lens calibration: FPN, FFC, DCP, undistortion

FastVCR Command-Line Application

Embedded systems rarely have a display attached, so FastVCR for Jetson is also available as a CLI application with the same capabilities as the GUI version. It covers headless scenarios — UAV payloads, remotely controlled cameras, unattended long-term recording and streaming — while keeping full runtime control over the sensor and the processing parameters. For remote preview, use the bundled RTSP player or VLC on the receiving side.

FastVCR is built from the modules of the Fastvideo CUDA image processing SDK, so the same GPU components are available for licensing and can be embedded directly into your own Jetson application. We also develop custom camera software to specification, including application-specific processing modes — describe your project in the contact form below.

Compatibility

• NVIDIA Jetson Orin NX and AGX Orin with CUDA 12.6
• NVIDIA Jetson Xavier NX and AGX Xavier with CUDA 11.4
• Legacy NVIDIA Jetson Nano and TX2 with CUDA 10.2
• MIPI CSI2 cameras with a driver that provides access to RAW frames and image sensor modes of operation

Software Downloads

• FastVCR build for MIPI cameras with the IMX477 image sensor on NVIDIA Jetson Orin — available upon request via the contact form below
• PDF manual for FastVCR software

Applications

• Robotics and embedded vision
• Drones and UAVs
• Automotive: ADAS, surround-view, and rearview cameras
• Industrial automation and surveillance
• Medical imaging: endoscopes and surgical cameras
• Data collection for AI applications with embedded cameras

FastVCR Roadmap

Recently released:

• Adaptive local tone mapping (ALTMapper) engine for single shot HDR
• NLM and bilateral denoisers on the GPU
• CUDA 12.6 support for NVIDIA Jetson Orin

In progress:

• Chromatic aberration removal in the RAW domain with self-calibration
• Compatibility with YOLO v5–v8
• Accelerated GPU kernels for better performance on Jetson Nano/TX2

Blog Posts on the Subject

• GPU software for machine vision cameras
• Jetson image processing framework
• Gpixel GMAX3265 image sensor processing
• GPU software for camera applications

About the author

Fyodor Serzhenko, PhD, is the founder and CEO of Fastvideo. He earned his PhD at the Moscow Institute of Physics and Technology (MIPT) in 1993. Since 2009 he has led the development of Fastvideo’s GPU-accelerated image codecs and ISP modules, including the Fastvideo SDK. Connect on LinkedIn.

Why you can trust these results

Fastvideo has built GPU-accelerated image processing software since 2009. Our codecs comply with the standards they implement — JPEG (ITU-T T.81 / ISO IEC 10918) and JPEG2000 (ITU-T T.800 / ISO IEC 15444) — and our tools are open source on GitHub. Every performance figure we publish is reproducible: see our benchmark methodology, download the Fastvideo SDK benchmark report (PDF), and measure it on your own GPU.