[Philip Lelyveld comment: this is the abstract. Contact the author at the link for more info]
[Fraunhoffer]
In cooperation with the Image & Video Coding Group, we have developed efficient 3D video coding methods for depth-enhanced 3D video formats, ranging from conventional stereo video (CSV) to multi-view video plus depth (MVD) with two or more views and associated depth data components. This format scalability is achieved by coding each video and depth map component by a conventional 2D video codec structure, based on the high-efficiency video coding technology (HEVC), as shown in Figure 1.
In order to provide compatibility with existing 2D video services, the base or independent view is coded using an HEVC codec. For coding the dependent views and the depth data, modified HEVC codecs are used, which are extended by including additional coding tools and inter-component prediction techniques that employ data from already coded components at the same time instance as indicated by the red arrows in Figure 1.
For supporting the decoding of video-only data, e.g. pure stereo video suitable for conventional stereo displays, the inter-component prediction can be configured in a way that video pictures can be decoded independently of the depth data, and the latter being discarded. For improving the coding efficiency of all video and depth components, additional tools have been developed, including:
- Coding of dependent views using disparity-compensated prediction, inter-view motion prediction and inter-view residual prediction.
- Depth map coding using new depth modeling modes, adapted motion compensation and motion vector coding, and motion parameter inheritance.
- Encoder control for depth-enhanced 3DV formats using view synthesis optimization with block-wise synthesized view distortion change and encoder-side render model.
- Decoder-side view synthesis based on DIBR, aligned with the encoder-side render model for overall coding and synthesis optimization for any multi-view display.
Dr. Karsten Müller
Head of 3D Coding Group
Tel. +49 30 31002-225
E-mail[1] H. Schwarz, C. Bartnik, S. Bosse, H. Brust, T. Hinz, H. Lakshman, D. Marpe, P. Merkle, K. Müller, H. Rhee, G. Tech, M. Winken, and T. Wiegand, “3D Video Coding Using Advanced Prediction, Depth Modeling, and Encoder Control Methods”, Proc. IEEE International Conference on Image Processing (ICIP’12), Orlando, Florida, USA, Oct. 2012.
[2] G. Tech, H. Schwarz, K. Müller, and T. Wiegand, “Effects of synthesized View Distortion based 3D Video Coding on the Quality of interpolated and extrapolated Views”, Proc. IEEE International Conference on Multimedia and Exposition (ICME’12), Melbourne, Australia, July 2012.
[3] H. Schwarz, C. Bartnik, S. Bosse, H. Brust, T. Hinz, H. Lakshman, D. Marpe, P. Merkle, K. Müller, H. Rhee, G. Tech, M. Winken, and T. Wiegand, “3D Video Coding Using Advanced Prediction, Depth Modeling, and Encoder Control Methods”, Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.
[4] P. Merkle, C. Bartnik, K. Müller, D. Marpe, and T. Wiegand, “3D Video: Depth Coding Based on Inter-component Prediction of Block Partitions”, Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.
[5] H. Schwarz and T. Wiegand, “Inter-View Prediction of Motion Data in Multiview Video Coding”, Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.
[6] G. Tech, H. Schwarz, K. Müller, and T. Wiegand, “3D Video Coding using the Synthesized View Distortion Change”, Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.
[7] M. Winken, H. Schwarz, and T. Wiegand, “Motion Vector Inheritance for High Efficiency 3D Video plus Depth Coding,” Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.
[8] S. Bosse, H. Schwarz, T. Hinz, T. Wiegand, “Encoder Control for Renderable Regions in High Efficiency Multiview Video Plus Depth Coding”, Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.
See the original post here: http://www.hhi.fraunhofer.de/de/departments/image-processing/3d-coding/3d-video-high-efficiency-coding/