News Stories

3D Video: High-Efficiency Coding

5.18.2012

[Philip Lelyveld comment: this is the abstract.  Contact the author at the link for more info]

[Fraunhoffer]

In cooperation with the Image & Video Coding Group, we have developed efficient 3D video coding methods for depth-enhanced 3D video formats, ranging from conventional stereo video (CSV) to multi-view video plus depth (MVD) with two or more views and associated depth data components. This format scalability is achieved by coding each video and depth map component by a conventional 2D video codec structure, based on the high-efficiency video coding technology (HEVC), as shown in Figure 1.

In order to provide compatibility with existing 2D video services, the base or independent view is coded using an HEVC codec. For coding the dependent views and the depth data, modified HEVC codecs are used, which are extended by including additional coding tools and inter-component prediction techniques that employ data from already coded components at the same time instance as indicated by the red arrows in Figure 1.

For supporting the decoding of video-only data, e.g. pure stereo video suitable for conventional stereo displays, the inter-component prediction can be configured in a way that video pictures can be decoded independently of the depth data, and the latter being discarded. For improving the coding efficiency of all video and depth components, additional tools have been developed, including:

  • Coding of dependent views using disparity-compensated prediction, inter-view motion prediction and inter-view residual prediction.
  • Depth map coding using new depth modeling modes, adapted motion compensation and motion vector coding, and motion parameter inheritance.
  • Encoder control for depth-enhanced 3DV formats using view synthesis optimization with block-wise synthesized view distortion change and encoder-side render model.
  • Decoder-side view synthesis based on DIBR, aligned with the encoder-side render model for overall coding and synthesis optimization for any multi-view display.

Dr. Karsten Müller
Head of 3D Coding Group

Tel. +49 30 31002-225

Publications

[1] H. Schwarz, C. Bartnik, S. Bosse, H. Brust, T. Hinz, H. Lakshman, D. Marpe, P. Merkle, K. Müller, H. Rhee, G. Tech, M. Winken, and T. Wiegand, “3D Video Coding Using Advanced Prediction, Depth Modeling, and Encoder Control Methods”, Proc. IEEE International Conference on Image Processing (ICIP’12), Orlando, Florida, USA, Oct. 2012.

[2] G. Tech, H. Schwarz, K. Müller, and T. Wiegand, “Effects of synthesized View Distortion based 3D Video Coding on the Quality of interpolated and extrapolated Views”, Proc. IEEE International Conference on Multimedia and Exposition (ICME’12), Melbourne, Australia, July 2012.

[3] H. Schwarz, C. Bartnik, S. Bosse, H. Brust, T. Hinz, H. Lakshman, D. Marpe, P. Merkle, K. Müller, H. Rhee, G. Tech, M. Winken, and T. Wiegand, “3D Video Coding Using Advanced Prediction, Depth Modeling, and Encoder Control Methods”, Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.

[4] P. Merkle, C. Bartnik, K. Müller, D. Marpe, and T. Wiegand, “3D Video: Depth Coding Based on Inter-component Prediction of Block Partitions”, Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.

[5] H. Schwarz and T. Wiegand, “Inter-View Prediction of Motion Data in Multiview Video Coding”, Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.

[6] G. Tech, H. Schwarz, K. Müller, and T. Wiegand, “3D Video Coding using the Synthesized View Distortion Change”, Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.

[7] M. Winken, H. Schwarz, and T. Wiegand, “Motion Vector Inheritance for High Efficiency 3D Video plus Depth Coding,” Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.

[8] S. Bosse, H. Schwarz, T. Hinz, T. Wiegand, “Encoder Control for Renderable Regions in High Efficiency Multiview Video Plus Depth Coding”, Proc. PCS 2012, Picture Coding Symposium, Krakow, Poland, May 2012.

See the original post here: http://www.hhi.fraunhofer.de/de/departments/image-processing/3d-coding/3d-video-high-efficiency-coding/

Augmented reality adds a new dimension to planning decisions

5.18.2012

[The Guardian]

Planning is the source of some of the fiercest debate in public life. Local residents often misunderstand how a new development will look when finished, or how it will affect the surrounding area.  …

By overlaying the real world with digital information, augmented reality software creates a geometrically accurate composite of the physical and the virtual. …

… AnEU-funded project called Arthur, which allowed architects and customers to review designs even before they were built, finished in 2004. It created computer-generated models on augmented-reality, head-mounted displays that enabled participants to view 3D virtual models on a planning table, tweaking their size and scale in real-time.

Urban Sketchers, another prototype device, uses a “mixed reality” interface. This allows parties interested in a proposed development to augment video footage of the site using sketching software. But new smartphone-based, augmented-reality planning tools could obviate the need for such specialist hardware. …

Read the full story here: http://www.guardian.co.uk/local-government-network/2012/may/18/augmented-reality-planning-applications

< PREVIOUS ARTICLES NEXT ARTICLES >

Specification for Naming VFX Image Sequences Released

ETC’s VFX Working Group has published a specification for best practices naming image sequences such as plates and comps. File naming is an essential tool for organizing the multitude of frames that are inputs and outputs from the VFX process. Prior to the publication of this specification, each organization had its own naming scheme, requiring custom processes for each partner, which often resulted in confusion and miscommunication.

The new ETC@USC specification focuses primarily on sequences of individual images. The initial use case was VFX plates, typically delivered as OpenEXR or DPX files. However, the team soon realized that the same naming conventions can apply to virtually any image sequence. Consequently, the specification was written to handle a wide array of assets and use cases.

To ensure all requirements are represented, the working group included over 2 dozen participants representing studios, VFX houses, tool creators, creatives and others.  The ETC@USC also worked closely with MovieLabs to ensure that the specification could be integrated as part of their 2030 Vision.

A key design criteria for this specification is compatibility with existing practices.  Chair of the VFX working group, Horst Sarubin of Universal Pictures, said: “Our studio is committed to being at the forefront of designing best industry practices to modernize and simplify workflows, and we believe this white paper succeeded in building a new foundation for tools to transfer files in the most efficient manner.”

This specification is compatible with other initiatives such as the Visual Effects Society (VES) Transfer Specifications. “We wanted to make it as seamless as possible for everyone to adopt this specification,” said working group co-chair and ETC@USC’s Erik Weaver. “To ensure all perspectives were represented we created a team of industry experts familiar with the handling of these materials and collaborated with a number of industry groups.”

“Collaboration between MovieLabs and important industry groups like the ETC is critical to implementing the 2030 Vision,” said Craig Seidel, SVP of MovieLabs. “This specification is a key step in defining the foundations for better software-defined workflows. We look forward to continued partnership with the ETC on implementing other critical elements of the 2030 Vision.”

The specification is available online for anyone to use.

Oops, something went wrong.