Understanding Video Compression and Raw Formats in Embedded Computer Vision

In the world of embedded computer vision, the choice of video compression and raw formats can have a significant impact on system performance, data transmission, and storage efficiency. Whether you’re working on real-time video processing in resource-constrained environments or handling large datasets for computer vision tasks, selecting the right format is crucial. This post will explore various video compression and raw formats and how they apply specifically to embedded systems.

1. Uncompressed Formats

In embedded systems, using uncompressed formats ensures no loss of image quality but often requires more bandwidth and storage capacity. These formats are typically used when precision is critical, and the system can handle large data sizes.

• NO_COMPRESSION_RGB
  The RGB format is widely used in scenarios where color fidelity is essential, such as object recognition or classification tasks. In embedded systems, RGB can be used when high-quality image data is needed for tasks that rely heavily on color accuracy. However, due to its large file size, it may be unsuitable for systems with limited memory or bandwidth.
• NO_COMPRESSION_RAW
  RAW formats contain minimally processed data directly from the image sensor, making them ideal for applications like high-precision machine vision or industrial inspection, where retaining every detail is critical. Embedded systems in these use cases must handle high data throughput, which may necessitate specialized hardware or optimized processing pipelines.
• NO_COMPRESSION_Y800/Y8/GREY
  Grayscale formats such as Y800, Y8, or GREY are common in computer vision tasks like edge detection, pattern matching, or depth sensing, where color data is unnecessary. In embedded systems, grayscale reduces memory and processing requirements while maintaining the essential information for algorithms that don’t require color.
• NO_COMPRESSION_Y16
  This 16-bit grayscale format offers higher dynamic range, useful in scientific or medical embedded applications where subtle differences in intensity matter. Systems using this format often have more advanced image sensors and require greater processing power.

2. Compressed Formats

Compressed formats are vital for reducing data size, especially in bandwidth-constrained embedded systems. These formats often balance quality and size, making them suitable for real-time video processing and transmission.

• UYVY/Y422/UYNV/V422
  These 4:2:2 chroma subsampling formats reduce the amount of color data stored while preserving the full luminance resolution, making them ideal for tasks where color is secondary to brightness or light intensity. In embedded vision systems, these formats allow for real-time video transmission with minimal visual quality loss. For example, a smart camera in a surveillance system can use UYVY to transmit video efficiently over a limited wireless connection.
• YUY2/YUYV/YUNV/YVYU
  These formats, which also use 4:2:2 chroma subsampling, are commonly deployed in real-time video capture and streaming. Embedded vision systems in robotics or autonomous vehicles can leverage these formats to process video streams with less computational overhead while retaining sufficient detail for object detection or tracking.
• AYUV_COMPRESSION
  AYUV format includes an alpha channel along with YUV data, which may be useful for tasks involving layered images, such as augmented reality or video overlays. In embedded systems, it can be applied in scenarios where transparency or alpha blending is required.

3. Planar Formats

Planar formats store the image data in separate planes for different color components. These formats are often more memory-efficient and are widely used in video processing pipelines in embedded vision systems, where memory bandwidth is limited.

• I420_COMPRESSION/IYUV_COMPRESSION
  Planar formats like I420 and IYUV are commonly used in video streaming and recording applications due to their efficient handling of YUV data. Embedded systems, such as drones or mobile devices, use these formats for real-time video encoding where bandwidth and memory limitations require compact data representation without significantly compromising quality.
• YV12_COMPRESSION
  Similar to I420, this format is ideal for embedded systems focused on video compression, such as smart security cameras. Its reduced chroma resolution allows for smaller file sizes, which is crucial when transmitting video over networks with limited bandwidth.
• NV12_COMPRESSION/NV21_COMPRESSION
  These formats offer interleaved chroma data, making them slightly more efficient than I420. They are commonly used in hardware-accelerated video processing pipelines, which are integral to embedded systems with limited processing power but dedicated video encoding hardware.

4. JPEG and PNG Compression

Still image compression techniques like JPEG and PNG are also relevant in the context of video processing, particularly when working with systems that process individual video frames.

• JPEG_COMPRESSION/MJPG_COMPRESSION
  JPEG compression is lossy but widely used for reducing file sizes. Motion JPEG (MJPG), where each frame in a video is a compressed JPEG image, is commonly used in embedded systems for video streaming or recording. This format is suitable for systems where moderate quality loss is acceptable in exchange for smaller file sizes and reduced bandwidth usage.
• PNG_COMPRESSION
  PNG compression is lossless and is used in scenarios where image quality cannot be compromised. For embedded systems that require high fidelity in image transmission—such as medical devices, satellite imaging, or industrial inspection equipment—PNG ensures that no data is lost during compression, though it requires more bandwidth and storage.

Applying These Formats in Embedded Computer Vision

In embedded computer vision systems, selecting the right format is critical for balancing the trade-offs between processing power, memory usage, and data fidelity. Here are some key considerations:

• Real-Time Processing: For real-time video processing in embedded systems like drones or autonomous robots, formats like UYVY, YUY2, or I420 are ideal due to their efficient use of bandwidth and processing power.
• Limited Storage or Bandwidth: Systems that need to store large amounts of video data or transmit video over constrained networks (e.g., IoT devices or remote sensors) benefit from formats like MJPG or YV12 that offer good compression while retaining useful visual information.
• High Precision Tasks: When precision is paramount, such as in medical or industrial vision systems, uncompressed formats like RAW or Y16 ensure that every detail is preserved. These systems must be designed with enough memory and processing power to handle the large file sizes that come with uncompressed data.
• Image Analysis and AI: For embedded systems running machine learning models on video feeds, grayscale formats like Y800 are often sufficient and efficient, as many AI algorithms focus on shape and structure rather than color. These formats reduce the data size while retaining the necessary information for inference.