(Philip Lelyveld comment: they extract the players and ball from the background, then perform free viewpoint mapping of the players onto a model of the stadium.)
KDDI R&D Laboratories has developed the world’s first technology for synthesizing and displaying 3D video of stadium sports in real time, from any angle chosen by the viewer. This technology was exhibited at CEATEC JAPAN 2010.
“Until now, the viewing angle for live soccer has only been changeable by switching between camera positions. This technology provides a new video experience, by offering views from angles where cameras can’t be installed, such as the middle of the pitch or in the air.”
“This screen shows the stadium as seen by eight HD cameras, and these are synthesized pictures from places where cameras aren’t installed, including aerial views, and shots from in among the players. On the other hand, this screen shows pictures from a single high-resolution camera, of the sort used for 4K rather than HD. This system creates pictures from any angle chosen by the viewer. Because the camera has such a high resolution, there’s relatively little loss of picture quality even if it zooms right in among the action.”
This technology uses a unique method to enable high-speed estimation of players’ positions on the field in 3D. It utilizes the fact that when soccer is broadcast, the image consists simply of players and a background.
“What viewers mainly want to see is the players. So we thought of technology for synthesizing pictures that look natural, by cutting out just the parts with players, and putting them against a plane. Then, the plane is transformed, in line with the direction or angle the viewer has chosen. The pictures you’re seeing now are being synthesized in real time using a single PC. So if you have a PC at home, this kind of 3D processing can be viewed immediately on a 3D TV.”
KDDI has also developed server technology that stores stadium 4K video content taken from many angles, then extracts and delivers pictures at high speed with low processing load. This achieves a real-time delivery system for pictures from any angle chosen by the viewer.
See the original post here: http://www.diginfo.tv/2010/10/19/10-0211-r-en.php