Convolution Neural Network based Video Coding Technique using Reference Video Synthesis

Abstract

In this paper, we propose a novel video coding technique that uses a virtual reference (VR) video frame, synthesized by a convolution neural network (CNN) for an inter-coding. Specifically, an encoder generates a VR frame from a video interpolation CNN (VI-CNN) using two reconstructed pictures, i.e., one from the forward reference frames and the other from the backward reference frames. The VR frame is included into the reference picture lists to exploit further temporal correlation in motion estimation and compensation. It is demonstrated by the experimental results that the proposed technique shows about 1.4% BD-rate reductions over the HEVC reference test model (HM 16.9) as an anchor in a Random Access (RA) coding scenario. © 2018 APSIPA organization.