Criminisi Algorithm for Image Inpainting

Application Background

In the field of image inpainting, two primary models are currently used: Partial Differential Equation (PDE)-based inpainting models [1-2] and texture synthesis-based inpainting models [3]. PDE models require substantial computation, are time-consuming, have limited texture restoration capability, and are only suitable for narrow area repairs like scratches, stains, and text removal. Texture synthesis models extract features from surrounding image areas and synthesize matching textures, making them appropriate for larger damaged regions.

Reference [4] separates images into structure and texture components, processes them using PDE methods and texture synthesis techniques respectively, and then merges the results. However, this method is only suitable for small-area repairs, relatively slow, and still suffers from blurring issues. To address these limitations, Criminisi et al. proposed an exemplar-based image inpainting algorithm in 2003 [5] that doesn't require image segmentation and simultaneously handles both texture and structure. Although this algorithm achieves satisfactory results, it consumes significant time and has deficiencies in priority and similarity calculations.

This paper improves upon the image inpainting algorithm from reference [5]. For more accurate priority calculation, we employ both gradient data terms and confidence levels to determine the filling order. To accelerate the inpainting process, we implement a local window search strategy. Finally, we use both color and gradient information to determine similarity, resulting in better visual quality. Extensive experimental results demonstrate that our algorithm improves inpainting efficiency while producing more satisfactory visual outcomes.

Key Technology

In exemplar-based image inpainting algorithms, the filling order is crucial. The priority function for determining filling sequence must consider two factors: the amount of known information in the template window and the structural characteristics around the target area. Blocks with more known information can utilize more surrounding data, while areas with prominent structural features contain rich structural information. Criminisi's priority function is defined as: P(p) = C(p) × D(p). However, when the isophote is perpendicular to the unit normal vector, D(p) = 0, meaning even if C(p) is large and only a few pixels are unknown in the entire block, the block cannot be filled promptly. This leads to unreliable priority calculation and affects inpainting quality. To solve this problem, our paper directly incorporates gradient information to compute block priorities. Point P is on the repair boundary dΩ, and ψp represents the block centered at point P.