Skip to main content

Accelerating innovative augmented reality broadcast applications through new industrial camera design

Periodic Reporting for period 4 - DBRLive (Accelerating innovative augmented reality broadcast applications through new industrial camera design)

Reporting period: 2018-08-01 to 2019-04-30

The aim is to simplify, miniaturize and scale the camera system for capturing real life videos for augmented reality applications. This is moree important along with change of Supponor’s (S's) business concept: from service company to technology development and licensing one. It allows less trained customer staff to manage field operations and faster growth of S's business. The main goals are support for the future 4K/UHD resolution in sports broadcasting, simplified effort in setting up and operating the camera system, and improved near infrared (NIR) imaging.

Compared to S's existing system, approx. 10-fold NIR imaging sensitivity and 40-fold dynamic range are achieved. Modern Internet protocol in data communication is employed.
Architecture of optics:
Two cases studied:
1. Solution with external (to TV camera) NIR imaging
2. Current architecture: Standard TV lens followed by beam splitter and a relay lens with iris to re-focus image for TV camera

Case 1 is too bulky and has parallax problem if there are obstacles in the field of view. Case 2 was chosen even though std. TV lenses cause veiling glare (flare) in NIR.

Optics:
Image size and lens mount for the optics in 4K TV sports cameras were studied: B4 will be the mount due to large depth-of-field (DOF). High dynamic range (HDR) NIR cameras is necessary. In HDR, NIR relay lenses are not necessary and NIR cameras run at full aperture resulting in higher NIR intensity and easier field operations but in reduced depth of field (DOF) which is not a problem according to S's experience.

Since NIR focus plane changes along with TV lens focus and zoom adjustments, an automatic NIR focusing was implemented.

Zeiss embedded the optical section of HDR camera in beam splitter which resulted in small size and cost. Two standard non-HDR cameras per NIR band are needed.

Testing of two image sensors was made. Cooling of NIR image sensors was assumed necessary, but the testing showed that dark noise of winning Sony sensor is low and cooling not needed.
Since Periodic Report 1 the technical requirement specification was completed, the solutions for technical challenges, such as suspension of the moving optical blocks, thermal management of the housing, implementation of HDR cameras have been found.

Since Periodic Report 2 the control, the hardware is working, except two minor manageable problems and long term reliability problem of relay lens. The lens system will be changed in the next version and the change will also result in improved image sharpness in corner areas. The mechanical construction is complete.

NIR imaging suffered from alignment problems of individual NIR images. Their reasons have been found, and the static alignment problem has been fixed by software. An improved camera baseplate has been found and will be used which greatly improves the dynamic alignment issue. Still further project is ongoing to make the construction more stiff.

The design of the first PFU prototype had serious problems due to wrong information in power supply manufacturer’s data sheet. Since then the construction has been completely revised. The design of PSU is working. The design of PFU, PSU and their interconnection were accepted by SGS safety specialist and once the next prototypes of PFU are available, they will be subjected to safety and EMC testing.

There are major subcontractor problems in software and firmware functions. This has prevented starting of field operations.

Overview of project results:

The architecture based on separate NIR lens(es) was rejected because it was found to be impractical. The architecture resembling the current one was chosen as it is a proven concept. The new HDR NIR cameras have a smart construction and high sensitivity. They can be focused independently of visible focusing.

Control, imaging and communications hardware have been implemented and are working as far as it has been tested. Also the the communication with the shock logger and the tree-dimensional accelerometer & gyroscope & magnetometer module are working now. The cement of two lens groups is subjected to too high stress at elevated temperatures. The cement and lens system will be changed in the next version to reduce stress level, and the change of lenses will also result in improved image sharpness in corner areas. The mechanical construction is complete, except two minor changes.



The design of the first PFU prototype had serious problems due to wrong information in power supply manufacturer’s data sheet. Since then the construction has been revised. The PSU is working. The design of PFU, PSU and their interconnection were accepted by SGS safety specialist and once the next production lot of prototypes of PFU and PSU are available, they will be subjected to functional and safety testing. The rated output voltage of PFU was increased from 210 V to 260 to overcome voltage dissipation problem in long cable between PFU and PSU.


Overview of the progress:

The project is severely delayed due to delay of one subcontractor and its period was extended to April 30th 2019. The third generation mechanical prototype was completed and thermal analysis was made.

The commercial contacts:

The National Hockey League invited Supponor to participate in World Cup in Toronto, in November 2016.
• Programme of demonstrations and customer development engagements with key stakeholders in German Football (2017 Roadshow with ADI followed by games at Frankfurt, Dortmund and Hamburg, resulting in formal approval by the German Football League)
• New product demonstrations and testing with Mediapro in Spain on new ‘NSA’ range of products, leading to launch of new product (for virtual carpet / goal mat replacement) in August 2017 and now growing in the current season
• NBA demos / testing in partnership with Dome Productions (the test Supponor did in May in Toronto) leading to the Summer League 2018 that Supponor did (Las Vegas, July 2018) – where the NBA broadcast Supponor’s virtual overlay solution on Tencent in China (their major distribution partner’s network)
• Upcoming… UEFA Champions League test in early November
• Football / FA: England v Costa Rica in June 2018
• NSA testing during LaLiga season at Eibar in 2018
• NBA Summer League 2018
• Mediapro – DBRLive 2018-2020
• Mediapro – NSA Virtual Carpets 2018-2020
• DFL – DBRLive 2018-2020, based on cost recovery and profit share, 59 games
• NBA All Star, Playoffs 2018/19 All Star Feb 2019-Rookie & Main All-Star games, Live virtual broadcast on Tencent, China, 2018/19 Finals May 30-June 11
Note:
• UEFA: Proof of concept only: 2 tests with in UEFA Champions League, Tottenham v Inter Milan 28th Nov 2018 and Barcelona v Lyons 13th March 2019.
The new HDR NIR cameras have a smart construction and high resolution, sensitivity. and dynamic range.Yet they can be focused independently of visible focusing. The space requirement is much less than with the existing solution. Due to smarter size and operation and improved performance, some new application areas of augmented reality may open up. The application in sports will become much easier and the operational staff can be much smaller. The education of people operating the system is smaller.