Multi-Camera Robotics: A Comprehensive Analysis

We identified 3 clear cases where engineers add one or more extra cameras:

1. Enlarging the Field of View
   In scenarios like mobile robots navigating through their environments, or in automated pick-and-place tasks involving multiple bins, the need to add a camera serves to increase the Field of View in order to detect more objects or obstacles.
2. Improving Resolution Over Field of View
   In cases where the Field of View is sufficient, but not the resolution, one can adjust the lens and/or the distance of the camera to the environment to narrow the Field of View, which increases the resolution, and add other cameras to compensate for the loss of Field of View.
3. Enhancing Robustness in Field of View
   In environments where reflections or occlusions occur, having several viewpoints can mitigate the risk of visual information loss. This redundancy ensures that even if one camera's view is compromised, others can compensate.

Multi-camera systems can quickly overload a robot's CPU due to the significant amount of data generated. This necessitates a shift towards more efficient processing methods. Utilizing GPU processing can be a more effective solution, as GPUs are better suited to handle large volumes of visual data. NVIDIA created the Isaac ROS ARGUS library to help offload some processing to the GPU. Additionally, selecting cameras that can generate depth maps internally (like the Intel Realsense) using their Visual Processing Units (VPUs) can significantly reduce the processing load on the robot's CPU.

When the object in view moves, or the camera moves, some kind of hardware-based synchronization (by means of an electrical signal/pulse ) is needed between the cameras. The cameras can be configured in two ways: One camera serves as the master, and the others act as slaves, synchronizing their capture times based on the master camera's signals. Alternatively, an external signal generator can be used, where all cameras are configured as slaves to this signal, ensuring synchronized data capture.

• Stereolabs ZED X Variants
  • These cameras are compatible with GMSL2 capture cards, which include trigger/strobe functions. They are well-suited for multi-camera systems on NVIDIA platforms.
• Intel Realsense Series
  • The D435, D415, D455, D456, and D457 models from Intel Realsense all have hardware pins to be master or slave trigger. Intel also wrote a White Paper on how to combine Realsense cameras.
• Ensenso Stereo Camera Series
  • Most Ensenso cameras are GigE cameras with a GPIO connector to trigger it externally.
• Leopard Imaging Hawk Stereo GMSL2 Camera
  • This camera is recognized for its high-resolution stereo imaging and compatibility with GMSL2 technology and the NVIDIA Jetson platform.
• ZIVID Two Structured Light Camera
  • Known for its precision in 3D structured light imaging, the ZIVID Two is fit for environments where detailed depth data is crucial and no motion nor FoV overlap is present.

• Leopard Imaging Maxim GMSL 2 OWL Camera
  • A GMSL camera that that is supported on the NVIDIA Jeston platform.
• Lucid Vision Labs Triton Camera
  • These GigE cameras come with trigger GPIO pins, allowing for easy integration into multi-camera systems.