Image matting is an essential technique to estimate the foreground objects in images and videos for editing and composition. The conventional deep learning approach takes the input image and associated trimap to get the alpha matte using convolution neural networks. But since the real-world input images for matting are mostly of very high resolution, such approaches efficiency suffers in real-world matting applications due to hardware limitations.
Deep learning-based ‘HD-Matt’: Image matting for high-resolution images
To address the issue mentioned above, HD-Matt, the first deep learning-based image matting approach for high-resolution image inputs, is proposed by a group of researchers from UIUC (University of Illinois, Urbana Champaign), Adobe Research, and the University of Oregon.
HD-Matt works on the ‘divide-and-conquer’ principle. More concretely, it works in a patch-based crop-and-stitch manner to matt high-resolution inputs such as 5000 x 5000 pixels. It crops the input into different patches and then estimates the alpha value of each of the patches. It uses a novel Cross-Patch Context module (CPC) to solve the issue of the prediction inconsistency between different patches. This module helps leverage cross-patch information for each current patch, thus reducing the information loss while using a single patch independently. Then comes the stitching part in which each patch’s estimated values are stitched together to produce the final alpha matte of the entire original image.
Methods in the section “Whole” take the whole image as input, whereas in the “Patch” section, patches of the image are given as inputs.
After several tests conducted on Image matting, it was found that the HD-Matt’s capability using the Adobe Image Matting (AIM) and Alpha Matting benchmarks was superior to other existing SOTA approaches. In other comparative evaluations with SOTA image matting methods like IndexNet and ContexNet, HD-Matt was able to extract more accurate details of the high-resolution images, with resolutions up to 6000 x 6000 pixels.
Paper: High-Resolution Deep Image Matting is on Arxiv, Deep Image Matting
Source: https://syncedreview.com/2020/09/22/adobes-dl-based-hdmatt-handles-image-details-thinner-than-hair/
Shilpi is a Contributor to Marktechpost.com. She is currently pursuing her third year of B.Tech in computer science and engineering from IIT Bhubaneswar. She has a keen interest in exploring latest technologies. She likes to write about different domains and learn about their real life applications.
