Realtime Panorama Stitching Pipeline on CUDA

Panoramic images are often used in applications that require wide field of view and high horizontal resolution. Application examples include surveillance, navigation, scenic views, sport, etc. In such systems video is usually captured with multiple cameras. The process of stitching is quite complicated and computationally intensive. We are working on realtime panorama video pipeline for 2/4-camera systems. We are utilizing Ximea 2K-4K cameras, each pointing at a different direction, to capture the wide field of view. These different views are overlapped to insure stitching to create panoramic video.

Calibration procedures

  • Dark frame calibration
  • Flat-Field calibration
  • Camera profile calibration with colorchecker
  • Distortion calibration with chess board
  • Scene calibration:
    • Keypoint computation and search with FAST or ORB feature detector
    • Keypoint analysis, correlation and matching
    • Homographies between pairs of images are computed using RANSAC
  • Composition of final stitching transform on CUDA, which also includes undistortion, homography, cylindrical projection, etc.

Output calibration parameters

  • Dark frame
  • Flat-Field map or frame
  • Digital Camera Profile (DCP)
  • Camera intrinsic parameters: principal point, aspect ratio, focal length, distortion center and coefficients
  • Camera extrincis parameters: translation, rotation, homography

Full pipeline for each camera

  • Image acquisition and data import to GPU memory
  • Dark frame subtraction
  • Flat-Field Correction
  • White Balance
  • Exposure correction
  • Debayer
  • Denoiser
  • Color correction with matrix profile
  • Curves and levels for RGB/HSV
  • 3D LUT
  • Projection together with undistortion
  • Crop and Resize
  • Sharpening
  • Final image blending
  • Gamma with output color space conversion
  • Compression to JPEG or JPEG2000

Contact Form

This form collects your name and email. Check out our Privacy Policy on how we protect and manage your personal data.