Skip to main content

Advanced VR, iMmersive serious games and Augmented REality as tools to raise awareness and access to European underwater CULTURal heritagE.

Periodic Reporting for period 1 - iMARECULTURE (Advanced VR, iMmersive serious games and Augmented REality as tools to raise awareness and access to European underwater CULTURal heritagE.)

Reporting period: 2016-11-01 to 2017-10-31

Project’s scope is to raise public’s awareness on European identity by focusing in maritime cultural heritage, which by default bridges different civilizations. In particular, iMARECULTURE aims in bringing inherently unreachable underwater cultural heritage within digital reach of the wide public by implementing virtual visits, serious games with immersive technologies and underwater augmented reality. Scope of the project is to design, analyze, develop and validate pioneer applications and systems in the context of Virtual Museums through collaborative and innovative research from a diverse group of scientists, researchers, archaeologists, experts and museums.
Underwater Cultural Heritage assets are widely spread into the Mediterranean. However, unlike land archaeological sites, submerged settlements, submerged ancient ports and especially shipwrecks, are not accessible to the general public nor even to all experts, due to their environment and depth. Photos and surfaced finds exhibited in maritime museums provide fragmented aspects of such sites but are the only ones visitors can actually see. Thus, Virtual Museums provide a unique opportunity for digital accessibility to both scholars and general public, interested in having a better grasp of underwater sites and maritime archaeology. Such digital encounters should provide virtual visit opportunities for children, elderly or people with mobility problems, and enhance the actual underwater visit with augmented digital content.
iMARECULTURE, envisages in changing the way public interacts with underwater CH. New methods and techniques proposed by the Project for maritime museums, could easily be adopted by any kind of CH museums. Project’s actions aim to raise public awareness through serious games, prior and after the visit to an archaeological site or museum. These actions will prolong museum visits, providing free serious games as pre- and after- visit experiences. In the museum, enhanced technologies, with state of art VR devices will provide a personalized dry visit experience to the visitor. After the visit visitors, who are intrigued, may extend the visit by being involved in a realistic virtual excavation game, using immersive technologies and HMD from his home.
Despite the recent developments in Augmented Reality in devices, tools and CH, efforts are limited to dry conditions. The main reason is the demanding environment in terms of 3D data acquisition and poor imaging conditions. Therefore augmented reality tools, which could enhance the digital experience of visitors in situ, do not work underwater. The Project will address this using hybrid tracking in underwater tablets, which will be provided to visitors.
So far, very few efforts have been done towards the creation of interactive, personalized and immersive ways to enhance general public’s understanding of the common European underwater and maritime culture and history.
The Projects aims in to provide content accessible to everyone, breaking the restrictions of time and space. New technologies such as VR and AR are excellent tools to increase European citizens’ curiosity and consequently understanding of maritime Cultural Heritage. On the long term this will also increase maritime museum visitors, while the Project is designed to provide new opportunities and tools to researchers, to shape access and study European Culture.
Work performed so far, was focusing in data retrieval and gathering. Such data will be either used within the serious games, or the dry visits with devices such as Head Mount Displays (i.e HTC Vive, Daydream VR, Cardboard), Holographic screens and Virtual Reality Caves. Data gathered include geodata for the seafaring game and a route minimization algorithm, which has been developed to predict the fastest routes among ancient ports.
Three ships namely, the Kyrenia, the Maggan Mikael and the Olympias trireme, have been digitally reconstructed, in extreme detail. Amphorae 3D models are also being gathered to form a database of amphorae types. The ontologies of the two 3D libraries, of ships and amphorae, are almost completed. Both ships and amphorae, will be used in the VR underwater excavation game, which is under development.
Significant work has also been done to prepare game development. A pilot hyper storytelling application has been produced and evaluated internally, to act as a base for further improvements and additions. The application scenarios have been finalized, and pilot applications of the games are under internal evaluation. Similarly, storytellers are drafting narratives and cut scenes for video production. Currently the cut scenes of the first game, the Seafaring, are under production.
The Seafaring game has undergone one major revision. The second version was internally evaluated and the second major revision is being planned. The underwater excavation VR game, has completed the digging module and the collaborative diving through telepresence. Three more modules of the game are under development. A working version of the VR 3D puzzles is also undergoing evaluation.
The detailed 3D models of the three test sites (Mazotos shipwreck at 45m, Xlendi shipwreck at 110m, and Baiae archaeological park in Baiae marine protection area) have been prepared for the dry visits. Texture has been processed to be compatible with VR devices. Two sets of texture will be available in each site are almost completed. One demonstrates the real underwater colour and another one with true colour.
In the AR department, which is mostly concerned with the precise localization of the underwater tablet, using hybrid acoustic and imaging techniques, several tests have taken place, in a variety of conditions. On these tests, real time colour correction is evaluated along with dehazing algorithms. Both these image enhancement techniques aim in providing improved view to the visitors in real time. Image enhancement tests also involve improved QR coded target recognition. Finally, AR tools have been tested for positioning in the underwater environment, to assert their influence on the hybrid tracking.
Project’s scientific contribution so far, includes:
- Algorithm for route estimation, based on real and dynamic weather conditions and sailing diagrams of ancient ships
- Automatic amphorae detection using CNN
- Ship and amphorae ontology improvements over CIDOR CRM
- Advanced hyper-storytelling approach
- Improvements on underwater virtual environment, such as lighting, fog, procedural seaweeds and fish movements.
- New dehazing technique for improved underwater target recognition
Project’s impact is also realized through the dissemination and publication list on conferences and journals. Further state-of-art results are to be expected, until Project’s completion.
Project’s impact is already apparent, since the coordinator and other partners have been invited in varying events to present the project. Considering that there are no solid results/deliverables released to public, so far, this is an important achievement. The impact is expected to be high, since there are going to be several open/free data/software/applications/services released by the project. Additional foreseen deliverables released to public, including a stand alone application of underwater image improvement, a service with route calculation, and tools for simplified ontology manipulation.
Amphorae automatic detection tool
Image enhancement evaluation on Baia site
Seafaring Serious Game
The Kyrenia pilot storytelling application
​First tests of the Underwater Augmented Reality Tablet of iMARECULTURE project (2)
​First tests of the Underwater Augmented Reality Tablet of iMARECULTURE project (1)
Kyrenia II 3D model
Underwater excavaton of amphorae
3D model of the ancient Athenian trireme Olympias
Route calculated for a point of origin and destination