Exemplar-Based Video Inpainting Device and Method for Spherical Panoramic Videos

360-Degree Panoramic Video Inpainting Technology Based on Spherical Projection Square Patches

This technology is an exemplar-based inpainting device and method that projects each location of a spherical panoramic video onto a spherical surface based on latitude and longitude, standardizes them into uniform square patches regardless of latitude, and then identifies and copies reference patches most similar to the priority pixels at the boundary of the damaged area.

Spherical panoramic videos are 2:1 rectangular projections of a spherical coordinate system, which causes subjects to stretch horizontally at higher latitudes, leading to significant errors when using standard planar image restoration techniques. Existing exemplar-based inpainting was limited to areas near the equator with minimal distortion, while pixel-based restoration methods struggled to reconstruct large damaged areas or repetitive patterns.

This technology uses a coordinate transformation formula to project onto a virtual sphere to obtain patches, then iteratively determines priorities, searches for similar patches, and copies pixels until the target area is fully restored, ensuring natural results that account for latitude-based distortion. It can be applied to 360-degree camera-based AR content and virtual tour video editing to seamlessly remove markers or unwanted objects from omnidirectional footage.

‍

‍

Key Features:

• A step of projecting the location of spherical panoramic video data onto a virtual plane using latitude and longitude to convert it into uniform square patch samples regardless of latitude
• A step of calculating the priority of all pixels on the boundary of the target restoration area to determine the first pixel with the highest priority
• A step of calculating patch similarity between all pixels in the reference area and the first pixel to select the most similar second pixel and copy it to the first pixel's location
• A square patch conversion unit that sets a virtual sphere and projects onto the spherical surface to mathematically interconvert square patch pixel coordinates and spherical panoramic video pixel coordinates

‍