Video compression is essential for managing the vast amounts of data generated by video streams, especially in embedded computer vision systems where bandwidth, storage, and processing power are limited. However, video compression introduces artifacts—visual distortions or anomalies—that can degrade image quality and hinder the performance of computer vision algorithms. In the context of artificial intelligence at the edge, these artifacts can significantly impact the effectiveness of machine learning models deployed on embedded systems. This article explores common types of compression artifacts, their causes, and their implications for edge computing AI applications.
What Are Video Compression Artifacts?
Compression artifacts are unintended side effects of the video compression process, where parts of the image are lost or distorted to reduce file size. These artifacts occur because lossy compression techniques like JPEG and H.264 discard parts of the video data to achieve a more compact file size. While this reduction helps minimize bandwidth usage or storage—crucial for AI on edge devices—it also compromises visual quality.
In embedded machine learning systems, compression artifacts can significantly impact the performance of computer vision algorithms and applications, such as object detection, tracking, or recognition. Let’s explore some common types of artifacts and their effects on machine learning at the edge.
1. Blockiness (Blocking Artifacts)
Description
Blockiness is one of the most common video compression artifacts, especially in highly compressed videos using lossy codecs like JPEG or H.264. It occurs when video frames are divided into small blocks, usually 8×8 or 16×16 pixels, and compression is applied to each block independently. This can result in visible block boundaries when compression is too aggressive, causing a “blocky” appearance in the image.
Causes
Blockiness happens when compression reduces the color and luminance data within each block to such an extent that neighboring blocks no longer blend smoothly together. It is particularly noticeable in flat or uniform areas of an image, such as skies or walls, where subtle gradients are lost during compression.
Impact
In embedded AI systems, blockiness can interfere with image processing tasks that rely on edge detection, such as shape analysis or feature extraction. The artificial edges created by blockiness may confuse computer vision algorithms, leading to false boundaries and reducing the accuracy of object recognition systems deployed on edge devices like the Raspberry Pi for machine learning applications.
2. Blurring
Description
Blurring artifacts occur when high-frequency details—such as edges, textures, or fine patterns—are lost during compression. The image appears soft or smudged, with details becoming indistinct.
Causes
Compression algorithms like H.264 or MPEG often prioritize reducing high-frequency data to save space, since human vision is less sensitive to small changes in fine detail. However, for machine learning on embedded systems, these fine details may be critical for accurate detection and classification.
Impact
Blurring can significantly reduce the effectiveness of algorithms that rely on sharp features, such as keypoint detection (e.g., SIFT or ORB algorithms). In edge machine learning applications, blurring can hinder the performance of machine learning models in tasks like facial recognition or license plate detection, where high-frequency details are necessary to distinguish fine features.
3. Color Banding (Quantization Artifacts)
Description
Color banding occurs when smooth gradients in an image are represented by a limited number of color shades, resulting in distinct bands of color rather than a smooth transition. This is particularly visible in areas like skies, shadows, or smooth surfaces.
Causes
Color banding is a result of quantization, a process during compression that reduces the number of distinct colors in an image. When the compression algorithm reduces color depth too aggressively, subtle transitions between colors are lost, leading to visible bands.
Impact
For embedded systems that rely on accurate color data—such as industrial vision systems or autonomous vehicles—color banding can interfere with color-based segmentation or classification tasks. Banding can confuse algorithms that use color histograms for object recognition or tracking, which is critical in AI edge computing applications.
4. Mosquito Noise (Edge Artifacts)
Description
Mosquito noise refers to the shimmering or flickering effect that appears around the edges of objects, especially in compressed videos. It is caused by compression algorithms that struggle to encode sharp edges accurately, leading to pixelation or noise near the edges.
Causes
This artifact often arises from lossy compression techniques that reduce detail in textured areas while preserving key edge information. The result is a “halo” of noise or pixelation around high-contrast edges, which becomes particularly noticeable during motion.
Impact
Mosquito noise can disrupt edge detection and tracking algorithms in embedded machine learning systems. For example, in a vision system designed to track moving objects—possibly running on devices like the AI Thinker ESP32-CAM—mosquito noise around object edges can create false positives or confuse the tracker, leading to inaccurate results in AI edge computing applications.
5. Ringing (Gibbs Phenomenon)
Description
Ringing artifacts manifest as faint ripples or halos around high-contrast edges, making them appear unnatural or wavy. This happens due to the limitation of compression algorithms in reproducing sharp transitions between different intensity levels.
Causes
Ringing is often caused by the discrete cosine transform (DCT) used in JPEG and MPEG compression. When strong edges or sudden changes in intensity occur, the DCT can introduce oscillations or ripples around the edges, known as the Gibbs phenomenon.
Impact
In embedded vision systems that rely on precise edge detection—such as those used in quality control or surveillance—ringing artifacts can blur the distinction between object boundaries and the background. This reduces the system’s ability to accurately localize and identify objects, affecting machine learning models deployed on embedded systems.
6. Temporal Artifacts (Motion Artifacts)
Description
Temporal artifacts are distortions that occur over time, often seen as ghosting or smearing in fast-moving scenes. These artifacts happen when compression reduces the frame rate or discards too much temporal information to save bandwidth.
Causes
Many compression techniques use motion estimation and compensation to reduce redundancy between frames in a video sequence. However, when compression is too aggressive, it can result in noticeable temporal artifacts where moving objects leave behind trails or appear jittery.
Impact
Temporal artifacts can severely affect motion detection and tracking algorithms in embedded AI systems. Systems such as autonomous vehicles or drones that rely on real-time object tracking may experience delays or inaccuracies in motion estimation, leading to poor decision-making in critical AI edge applications.
Mitigating Compression Artifacts in Embedded Systems
While compression artifacts are often unavoidable, there are strategies to mitigate their impact on machine learning embedded systems:
Optimize Compression Settings
Use a balance between compression level and video quality. For real-time systems, applying moderate compression levels (e.g., using 4:2:2 chroma subsampling instead of 4:2:0) can help preserve essential details while reducing data size. This balance is crucial in edge computing AI, where resources are limited but performance is critical.
Post-Processing
Algorithms such as deblocking filters, denoising, or super-resolution techniques can reduce the visibility of compression artifacts after decoding. Embedded systems that can afford additional processing steps—such as those powered by Raspberry Pi for machine learning applications—can apply these techniques to improve image quality.
Use Hardware-Accelerated Codecs
In embedded systems, using hardware-accelerated codecs can improve video quality without increasing latency or processing overhead. Many embedded processors, including those used in AI edge computers, come with built-in support for hardware-accelerated encoding and decoding, which can reduce compression artifacts.
Choose Lossless Compression for Critical Tasks
When image quality is paramount—such as in medical or scientific applications—consider using lossless compression formats like PNG, which preserve every detail at the cost of larger file sizes. For edge machine learning applications where data integrity is critical, lossless compression ensures that machine learning models receive the highest quality input.
Conclusion
Understanding and mitigating video compression artifacts is essential for the successful deployment of machine learning on embedded systems. As artificial intelligence at the edge continues to grow, with applications ranging from autonomous vehicles to smart cameras using AI Thinker ESP32-CAM or Raspberry Pi for AI, addressing these artifacts becomes increasingly important. Computer vision algorithms are sensitive to input data quality, and compression artifacts can significantly degrade performance.
For organizations looking to implement AI in embedded systems, partnering with experts in machine learning consulting services can help navigate these challenges. A well-planned AI strategy consulting can ensure that your edge computing AI applications are optimized for both performance and resource constraints.
By carefully balancing compression settings, utilizing post-processing techniques, and leveraging hardware-accelerated codecs, embedded systems can effectively manage video data without compromising the performance of computer vision algorithms and applications. As machine learning and embedded systems continue to converge, addressing video compression artifacts will remain a key consideration in the development of robust AI edge solutions.