Final Report Summary - VISIONAIR (VISION ADVANCED INFRASTRUCTURE FOR RESEARCH)
Executive Summary:
VISIONAIR is organized through a technological oriented vision. Our 29 facilities were associated to four technological domains: 1/ Augmented collaborative environments, including holography, were supported to enable local or remote collaboration of actors. 2/ Ultra-High definition and Networking facilities provided UHD displays (4K, 8K even in 3D), including streaming the corresponding content on networks. 3/Virtual Reality facilities are related to CAVEs and other immersive environments where the user is immersed in virtual scenes. 4/ Scientific visualization facilities were dedicated to the navigation and understanding of results issued from High Performance Computation or from nature and physical observation.
VISIONAIR hosted 122 external projects from the start of 2012 to the start of 2015, covering a wide spectrum of disciplines as:
• Improvement of Engineering processes: interacting with fluid dynamics simulations, weather forcast, new style design, extended vision and perception of technical 2D drawings, manufacturing process analysis or simulation, augmented tools for manufacturing, manufacturing plant layout arrangement, training on assembly and disassembly of products.
• Remote collaboration: remote physical feedback by handling haptic device from a long distance, co-working sharing 4K panels, co-organisation of nurse schedule, surgery tele-operation; and also remote collaborative concerts involving musician and dansors located in various countries and participating to a same concert within a virtual environment.
• New tools and methods for medical applications. Ergonomy analysis to support sport analysis activities. Training gun fighters, analysing rugby-man or basket-ball gestures or enabling new protocols of remote interconnections between athlete and coaches. It was efficiently demonstrated the interest of 3D ultra-high definition quality with very low bandwidth and very low latency over networks to support this kind of remote collaboration.
• Museum applications were realized. The Last Supper from Leonardo Da Vinci was reconstructed in 3D to extend its perception, reconstruction of fragile historical sites for patrimony conservation and archeological studies.
• We also organized a virtual visit of Mars with data incoming from observatories, exploring seas and animals scanned from tomography and so much more including studies about virtual environment perceptions, pyschology, etc.
VISIONAIR is very proud of all this variety of activities which consolidated and extended a community practising the newest visualization and interaction technologies.
Project Context and Objectives:
The VISIONAIR infrastructure is a network of high level visualization and interaction facilities available for the European Research community. These facilities require expensive investment, maintenance and preparation work to enable innovative usage. VISIONAIR expects to share this effort made by teams in various location spread in Europe to offer more access to scientists and users who cannot make the corresponding investments. The partners of VISIONAIR were issued from several communities represented by FP6 network of excellence (VRL-KCIP, Intuition, aim@shape) but also from partners in connection with other infrastructures (HPC-Europa, Network providers, etc).
When starting this infrastructure in 2011, VISIONAIR was clustering 24 partners involved in 29 facilities. These facilities were mainly disconnected. Visualization and Interaction technological domains remain poorly standardized. For 3D visualization, most partners are either using some commercial solutions or developing their own software solution to support use-cases to be achieved. University of Stuttgart (HLRS) developed the COVISE engine; Sztaki developped VIRCA, INRIA Rennes developped OpenMask. Grenoble-INP developped CVE and no commercial solutions (Unity, 3DVia-Virtools) or open-source project (Blender, Ogre) can be considered as a standard. Visualization and Interaction is also supported by Ultra-High definition and streaming technologies and if the market defines standards (Full HD, 4K, incoming 8K, JPEG2000 protocol), here again every use case cannot be achieved out of the box. Complex and adapted connections must be engineered. Last but not least no real repository was existing to share models and tools about this wide domain. Because the 29 initial facilities were disconnected, the partners were mainly creating local demonstrations about training, expert assistance, telepresence, collaboration, streaming, cognition issues within immersion but the creation of benchmark use case and comparison data base was just a will. A few interactions between facilities were organised through various projects but the result were hardly consolidated.
VISIONAIR has been first organized through a technological oriented vision. Our 29 facilities were associated to four technological domains:
• Augmented collaborative environments, where every augmented reality technology, including holography was supported to enable local or remote collaboration of actors.
• Ultra-High definition and Networking facilities provided UHD displays (4K, 8K even in 3D) which was the first facilities in Europe at this level of resolution, but also streaming the corresponding content on networks.
• Virtual Reality facilities are related to CAVEs and other immersive environments where the user can immersed in virtual scenes.
• Scientific visualization facilities were dedicated to the navigation and understanding of results issued from High Performance Computation or from nature and physical observation.
The knowledge about these facilities was unclear. VISIONAIR has then developed an E-Resource map which is the first inventory of the 29 facilities. It is a first important result of VISIONAIR. Indeed such an action has two effects. Obviously, it creates an awareness capacity because the devices can be exposed to a wide public but it expected also to create an ontology of resources which drives a common vision towards a standard classification of devices. This E-Resource map is operational and could be open to a wider access.
The E-Resource map also promotes the facilities towards external interested users. As an infrastructure a main goal of VISIONAIR is to open access to the best researchers from any domain of competencies. The TransNational Access model (TNA) was a major activity. A permanent call for project was organised from January 2012 to end of January 2015. A committee review was in charge to assess the submitted project with a peer review process including a technical review about feasibility and two scientific reviews: one from an internal member of the consortium and one external review from the discipline of the application. During this period VISIONAIR registered in its database 208 projects and 122 projects have been successfully completed by the end of the project. That means 122 new use-cases were experimented with the support of VISIONAIR. All the completed projects were reported in the Visionair Application Management (VAM) database with a minimal format including the question of research, a presentation of the achieved process and the main conclusions of the project.
With TNA projects, VISIONAIR demonstrated its capacity to support any discipline and to become a major location for multidisciplinary research activity. VISIONAIR covered a wide spectrum of disciplines with:
• Engineering applications: immersion in fluid dynamics simulation for engineering improvement of product design, immersion in weather forecast, style design activities for artists and designers, extended vision and perception of technical 2D drawings, manufacturing process analysis or simulation, augmented tools for manufacturing, remote collaboration around the arrangement of a manufacturing plant layout, training assembly disassembly of products. But engineering needs often collaborative practises: handling haptic device from a long distance, co-working in front of 4K shared panels, co-organisation of nurse schedule, tele-operation for medical solutions.
• New tools and methods for medical applications. In this field ergonomics is a traditional activity including visualization and interaction devices. But VISIONAIR was also expected to support sport analysis activities. Training gun fighters, analysing rugbyman or basketball gestures or enabling new protocols of remote interconnections between athletes and coaches. It was efficiently demonstrated the interest of 3D ultra-high definition quality with very low bandwidth and very low latency over networks to support this kind of remote collaboration.
• Artists were also motivated to use VISIONAIR facilities. In ICT Vilnius in December 2013, VISIONAIR supporting the SPECIFI project demonstrated its capacity to create a remote collaborative concert involving a pianist in Barcelona, players in Poznan, and one last musician in Vilnius. They played together in real-time with a dancer improvising in Grenoble. The dancer was in a CAVE, and a 3D avatar was created in real-time allowing the synthesis of a 3D scene for the audience in Vilnius. Furthermore some applications were realized to improve story telling around the last super from Leonardo Da Vinci in collaboration with the Last Super Interactive project thus leading to museum innovative applications.
• About museums, VISIONAIR was also involved about patrimony conservation creating virtual mock-up of fragile historical sites or organizing archaeological studies in virtual environments.
• Architecture applications, support to mathematicians who expected to visually check some mathematics assumptions, but also virtual visit of Mars with data incoming from observatories, exploring seas and animals scanned from tomography and so much else including studies about virtual environment perceptions, psychology, etc.
VISIONAIR is very proud of all this variety of activities which consolidated and extended the community. The European Celtic excellence price discerned to the HyperMed TNA project came to consolidate the proudness.
An internal seminar where the 24 partners were invited on a regular base (either every two weeks or monthly) was also organized. This seminar was the opportunity to share common knowledge and to improve the strength of the emerging VISIONAIR community. In the first year (2011), when no TNA project was applied the seminar was the opportunity to discover the VISIONAIR facilities. Then starting with the first TNA projects, the seminar turned into technical tutorials to demonstrate specific use of the facilities than to present the TNA results. Every seminar presentation was recorded and is available from a broadcast server: it delivers 65 tutorials. This is an important networking activity which created a capitalized output. Reports and recorded tutorials are a reach database of benchmarks and use-cases which is shared widely from our website.
VISIONAIR makes sense as soon as its TNA projects are organised and capitalized for potential future re-use. The ontology issued from the aim@shape project, initially dedicated to share shape models, was extended to visualization and interaction domain and was also connected to the Eresource map. Then tools referred in the new VVS ontology (VISIONAIR Visualization System) are directly associated with the facilities which deliver them. New protocols were also implemented and tested to enable remote usage of the corresponding algorithms. To avoid an increasing number of information sources every information from the VAM database (TNA projects), from VVS system, from the Eresource Map were merged in the VISIONAIR Browser which acts as a single entry point to all the VISIONAIR results
(http://www.infra-visionair.eu/what-we-offer/visionair-resources.html). VISIONAIR browser provides first access to content through a discipline filter. From a discipline the corresponding facilities, TNA projects and adapted tutorials are proposed. But the browser offers direct access to the TNA projects, tutorials and resources (facilities). The list of available service are also directly listed there. The summary of joint research actions (internal research combining several partners) and information about our club of partner are available there.
The Browser demonstrates the capacity of VISIONAIR to create synergies and enable new type of interconnections. The facilities were initially mainly disconnected but the VISIONAIR project was a unique opportunity to interconnect them. Here things are not solved: interoperability is still an issue indeed but the partners learned and experimented a lot during the past period. Many TNAs expected the support of several facilities as the Art Concert experience mentioned previously. TNAs enforced the movement which was also investigated within the Joint Research Activity. How to design a manufactured product in Barcelona and to get a 3D feedback on an holographic display in Stockholm. How to connect an immersion CAVE in Rennes and a second one in London, or between Image Institute in Chalon sur Soane and another CAVE in Madrid, how to process information from various sources? Many demonstrators were developed and created new reference knowledge for these activities. Indeed by the end, VISIONAIR developped a real networking knowledge. Joint research actions were decomposed into 3 work-packages. One about hardware, one about tools and methods, and one about assessment of the usage and impact of technologies. Twenty internal research actions have been completed, extending the overall knowledge of the VISIONAIR group. The link between engineering CAD models and visualization and interaction was investigating creating new services enabling the automatic recognition of kinematics joints in CAD model to prepare virtual worlds. The orchestration of remote algorithms was also successfully tested, new 3D and 2D metaphors were compared and classified respect to usage, new algorithms to improve real-time remote presence, new methods for large navigation in restricted immersive environments, new tangible interactors were investigated and tested. Usage of newest holographic table with remote connection was demonstrated and completed the activities about interoperability among distributed platforms. Modeling (and support to) user requirement and user engagement, assessment of collaboration work, use of model driven applications to develop collaborative environment were also investigated in JRAs. All these activities were directly complementary to the needs observed within TNA projects and JRAs experimented new services that could be developed or become operational for new applications.
The community is still strongly alive. This knowledge must be maintained, developped and opened. The partners of VISIONAIR contacted in many directions other infrastructures and communities to go ahead. A Special Interest Group of the EURO-VR association was created in December 2015 to support this activity while opening it to a wider participation. The intention remains to build technical interconnections between distant facilities to open access to new applications. Visualization and interaction must become a core service as internet networks are considered now. Over the network end-user need tangible, intuitive, and high quality visualization and interaction models. There is thus an intermediary layer of tools, methods that must be maintained and developed. VISIONAIR is clearly a first instantiation of such a layer. VISIONAIR has been in direct contact with HPC-EUROPA, PRACE, EMIRACLE, CIRP, Design Society, SPECIFI, VFF (Virtual Factory Framework), EPOS (european planet observing system), CINEGRID, etc.
No doubt that the story is not ended!
Project Results:
Overall presentation of S & T results
As an infrastructure the main goal of VISIONAIR was to support TNA projects. 122 projects were completed supporting external researchers who expected to use the VISIONAIR facilities for their projects. It is our major result with a lot of steps passed in many fields of research. To enable these TNA activities, coordination actions expected to solve some technical issues supporting the consortium integration. These developments were mainly conducted in the work-package WP3. In addition the joint research activities are sources of science and technological results. They were organised in 20 actions to pass over technical and scientific challenges which will improve the VISIONAIR support to TNAs.
This part follows the work-package organisation to report the main scientific and technological results. For many actions the inter-relations between work-packages was important: some TNA use-cases were directly re-used in Joint Research Activities which were enabled by coordination actions. This may lead to a few repetitions in the following sections. First we present the coordination activities results. Then the Trans-National access results are reported to finalize with joint research activity reports.
Coordination activities
Scientific and technical issues within coordination activities were associated to work-package 3. This work-package was organized into 4 tasks dedicated to improve technical and coordination integration within the consortium.
o T3.1.1: Video-conferencing/communication:
The task 3.1.1 was in charge to ensure communication between partners. Many solutions exist and have been clearly improved during the VISIONAIR project duration. It was not so easy four years ago to animate a project as VISIONAIR with an important usage of video-conference systems. The various solutions were benchmarked within this task and the EVO system was selected. It was not the better quality system but it was free for scientific communities and had the big advantage to be ported on most platforms. Indeed it was encoded within a java platform independent solutions.
At the end of year one the EVO providers changed their business model to turn it into a commercial solution, named SeeVogh. A new benchmark was operated inside VISIONAIR. Because RENATER was supporting a SeeVogh Community, and because Grenoble INP is a referenced user of RENATER, it was possible to provide access for VISIONAIR partners to the RENATER community within SeeVogh servers. During period 2 and 3, Grenoble INP created local accounts for every partner to provide access to the commercial solution SeeVogh which is deployed by RENATER. The usage of the SeeVogh network works in a satisfying way; no additional efforts are (or need to be) employed. Although SeeVogh has a more commercial approach as compared to the EVO solution, the current working method is certainly feasible for all Visionair project partners. Meetings, seminars and presentations run smoothly, almost without being hampered by technical issues.
If SeeVogh was satisfactory we still experienced technical issues with some partners. Mainly due to a lack of preparation of the work stations. It is a state-of-the-art result. The technology had an impressive enhancement in the last four years but the connections must be checked, specific loudspeaker or microphone must be used to avoid echo and disturbance.
It must be noted that most coordination activities where coordinated through video-conference, saving a big budget and time for travels. It is thus a very positive experience where the VISIONAIR consortium demonstrated its capacity to use intensively these solutions which is technical result. We just learned that RENATER stopped its participation to SeeVogh but our benchmark analysis let us with plenty alternative solutions with a growing maturity.
o T3.1.2: Broadcasting techniques:
Broadcasting techniques were not new. The main technical issue was to define a process to use existing solutions for the promotion of our internal results. No additional effort need to be employed unless to follow the process as soon as we decide to publish a video.
We intensively practised broadcasting. Some TNA projects were reported via videos and the internal seminars conducted by video-conferences were recorded and reworked to provide a publishable video. These video tutorials are created by UPatras. Grenoble INP was in charge to upload the videos on the University of Grenoble broadcasting server. Then the link is shared and deployed on the website. In parallel, Videos were posted within YOUTUBE to ensure a wider promotion of VISIONAIR actions.
Here again it is a good success of the VISIONAIR activity. More than 65 videos from 5 minutes to one hour were produced and are available from the VISIONAIR website and usually in the VISIONAIR group in YOUTUBE.
o WP3 T3.2 Synthetic environments:
A key point about visualization and interaction technologies is that they are not fully plug and play. Every use case requires new development and the potential to imagine new use-case is very high. An infrastructure supporting access to as set of heterogeneous facilities needs new adaption and developments for every new use-case. It must be a collaborative process involving the final user and the staff in charge of connecting the technical resources. The University of Twente developped the concept of synthetic environment. It is indeed a road-map system taking in charge the main steps of the collaboration from the initial demand to the delivery of the specified system.
A web tool was thus developped and open for usage by TNA-projects. The employment of the roadmap has been demonstrated during the General Assemblies and during a VisionAir seminar.
A significant number of partners indicated advantages of the roadmap but most partners still apply this process in a more informal mode. Some best-practices knowledge have been integrated in the synthetic environment roadmap; Given the number of page hits, many partners do take inspiration from the roadmap, but do record their prepararory processes in a different manner. The road-map contributes to the best-practices being availabe and takes away the fear that the first few best-practices attract unbalanced amounts of attention. The road-map remains a service available from the VISIONAIR portail for any other project.
o T3.3 : Interoperability issues
Interoperability challenges are still very important. The creation of an infrastructure like VISIONAIR was obviously facing the corresponding traditional issues. First because the facilities are spread in Europe and were not initially connected. VISIONAIR project was a unique opportunity to interconnect them. Interoperability is still an issue indeed but the partners learned and experimented a lot with VISIONAIR. Many TNAs expected the support of several facilities. In ICT Vilnius in December 2013, VISIONAIR supporting the SPECIFI TNA project, demonstrated its capacity to create a remote collaborative concert involving a pianist in Barcelona, players in Poznan, and one last musician in Vilnius. They played together in real-time with a dansor improvising in Grenoble. The dansor was in a CAVE, and a 3D avatar was created in realtime allowing the synthesis of a 3D scene for the audience in Vilnius. Indeed TNAs highlighted the demand and in many case Joint Research actions were expected to solve this issue. How to design a manufactured product in Barcelona and to get a 3D feedback on an holographic display in Stockholm. How to connect an immersion CAVE in Rennes and a second one in London, or between Image Institute in Chalon sur Soane and another CAVE in Madrid, how to process information from various sources?
Benchmarks were experimented. Arts et Métiers Paris Tech provided a full CATIA model of a complete real plant. The challenge was then to find the best way to import this very huge model on various Visualization sites. Several guidelines and tools were developed by HLRS and Grenoble-Inp about this case and overpassed the challenge.
Another usual interoperability barrier concerns the internet connections of two sites because of security issues. A VISIONAIR virtual private network was implemented by I2CAT to offer network connections free of firewall issues and was provided as a technical service for any need.
Many demonstrators were developed and created new reference knowledge for these activities. Indeed by the end, VISIONAIR developped a real networking knowledge. The first results highlighted the existence of many solutions to ensure interoperability between facilities. Nevertheless, use cases based on real scenario have shown some issues in using those solutions. Guidelines were produced for several specific domain where transforming domain data to visualization environment is expected.
o T3.4 : Virtual service infrastructure adaption for visionair needs
This task mainly concerned the evolution of the VVS service that is operated within WP8.
The VVS system is an evolution of Digital Shape Workbench, an ontology-based repository of shapes, tools and ontologies developed in the aim@shape FP7 network of excellence, coordinated by CNR. The DWS server was moved the CNR facilities and its maintenance was taken in charge by CNR. Within VISIONAIR, the shape and tool ontologies were extended to cope with Visualization and Interaction issues, the search and browsing capabilities were extended and improved turning the system into a Virtual Visualization System.
In WP1, a E-Resource map was developped by University of Patras. The role of this service was to deliver a catalog of all the VISIONAIR resources. This second ontology system is a first opportunity to declare, and promote the technical tools owned by the partners and it becomes a reference data-base in Europe.
Both CNR and Upatras collaborated to interconnect the two ontology. Especially the software resources owned by the facilities are shared by the two systems and are synchronized accordingly.
Technically VVS is first a huge repository of 3D shapes. It provides capabilities to enable content-based retrieval of 3D shapes. Hence, without using meta-data description, VVS may find similar shapes by direct comparison of models. The data repository was extended with new models including the creation of a specific manufacturing oriented view of the shape repository, thanks to the ontology defined in the JRA in WP9 which also guarantees connections to the VFF IP project results working to the definition of a Virtual Factory Framework.
VVS is also a catalog of algorithms and tools. Some joint research actions extended the VVS with a new repository of shape processing workflows to both provide informative material and to include the possibility to directly executing shape processing operations as services. .
In the last year the number of access to the VVS was doubled showing the interest for the platform and the results of a good dissemination of the VVS and of VISIONAIR. We can also note that the % of increment of visits of the shape repository (132%) is much bigger than the tool and ontology repositories (respectively 90% and 84%) showing the large interest in reusing shapes. Moreover most of the accesses are from Europe (51,43% for the shape repository, 63,66% to the tool repository and 56,51% to the ontology repository according to ClusterMaps© and 49% according to Google Analytics) followed by Asia and then North America.
These results demonstrate the high level quality of VISIONAIR services. Some over services were built and delivered from the website. To maintain a single portail to these services, Grenoble INP developed and maintains the VISIONAIR Browser which provides an integrated access to all the services and create relations between the resources in the E-Resource Map, the Trans-National Access Project, the VVS system, the broadcasted tutorials and promotion videos and the results of joint research activities.
Trans-National access results
The main result about Trans-National Access is that VISIONAIR supported 122 projects during the 3 years were TNA support was active. The ramp-up has been quite important. VISIONAIR was in a way a new infrastructure even if facilities were already existing before VISIONAIR. Before VISIONAIR most partners did not practise a kind of service activity and they were more involved in research. The first year was - as anticipated in the initial description of work - a period of adaptation of the corresponding processes. It was a hard but very positive process for every partner. The creation of the committee review, the definition of rules and criteria to apply and accept projects, the process to capitalize what is done and considered as completed, the capacity to better formalize a facility activity is a real added value for most partners.
Then following the first call for project the first proposals arrived. 6 projects were completed during the last 6 month of period 1. We faced some mus-understanding about the regulation to fund projects coming from outside Europe and it was mandatory to revise our promotion to remove access from external applications. At the end of period 2, 18 months later, 50 new projects were completed and during the last 12 months, 67 projects were completed. The acceleration was obvious. It can be noted that we received a set of applications which were either continuation of a past project or submission of a new project from the same applicant. These cases demonstrate that the corresponding applicants were convinced by the VISIONAIR support.
At the end of the project we were in a stage when it became mandatory to reject some incoming projects because of the deadline, and to cancel some accepted projects because we did not have enough time any more to schedule the projects. We still receive demands, three months later, and we have clear indications that some projects initiated in the format of TNAs go ahead outside VISIONAIR model.
Many projects were very long to start. During the VISIONAIR program submission we sub-estimated the effort expected to prepare projects. Several TNA projects expected 2 years of remote collaboration before the formal invitation of guests. This long process was due to un-maturity of technologies and the necessity to adapt the facilities to the demands. This fact mainly explains the big activity in the last year when project were now read. The authorization from the last amendment (covering the last period) to take into account some preparation unit of access was a good option to measure a real effort associated to every trans-national access project.
The European Celtic excellence price discerned to the HyperMed TNA project demonstrates that VISIONAIR has been recognized through its support action, Obviously the VISIONAIR consortium is proud of this award, but also proud that VISIONAIR supported a very wide variety of disciplines and challenges. We covered :
Engineering applications: immersion in fluid dynamics simulation for engineering improvement of product design, immersion in weather forecast, style design activities for artists and designers, extended vision and perception of technical 2D drawings, manufacturing process analysis or simulation, augmented tools for manufacturing, remote collaboration around the arrangement of a manufacturing plant layout, training assembly disassembly of products. But engineering needs often collaborative practises: handling haptic device from a long distance, co-working in front of 4K shared panels, co-organisation of nurse schedule, tele-operation for medical solutions.
New tools and methods for medical applications. In this field ergonomics is a traditional activity including visualization and interaction devices. But VISIONAIR was also expected to support sport analysis activities. Training gun fighters, analysing rugbyman or basketball gestures or enabling new protocols of remote interconnections between athletes and coaches. It was efficiently demonstrated the interest of 3D ultra-high definition quality with very low bandwidth and very low latency over networks to support this kind of remote collaboration.
Artists were also motivated to use VISIONAIR facilities. In ICT Vilnius in December 2013, VISIONAIR supporting the SPECIFI project demonstrated its capacity to create a remote collaborative concert involving a pianist in Barcelona, players in Poznan, and one musician in Vilnius. They played together in real-time with a dancer performing in Grenoble. The dancer was in a CAVE, and a 3D avatar was created in real-time allowing the synthesis of a 3D scene for the audience in Vilnius. And that's not all, some applications were realized to improve story telling around the last super from Leonardo Da Vinci in collaboration with the Last Super Interactive project thus leading to museum innovative applications.
About museums, VISIONAIR was also involved about patrimony conservation creating virtual mock-up of fragile historical sites or organizing archaeological studies in virtual environments.
Architecture applications, support to mathematicians who expected to visually check some mathematics assumptions, but also virtual visit of Mars with data incoming from observatories, exploring seas and animals scanned from tomography and so much else including studies about virtual environment perceptions, psychology, etc.
This list could be extended with the fine description of the 122 completed projects, but the main important result is the capitalization of all these new use-cases. Visualisation and Interaction is definitely a trans-disciplinary resource which remains less mature than network facilities and thus require more support to enable a specific use case. The capitalization of the use cases creates a knowledge data-base that will remain published on the VISIONAIR portal for any reuse, comparisons and enhancement.
The trans-national access activities were split into four work-packages for management convenience. This four work-packages are presented hereafter. The boundaries between work-packages was not closed and indeed it was often difficult to associate the projects to the work-packages. This surely mean that this convenient decomposition which was originally due to sub-communities between partners does not make much sense or that VISIONAIR succeeded in a good integration of its internal communities.
• Scientific Visualization
Scientific Visualisation encompasses a wide area of fields including – but not limited to – engineering, medical visualisation, chemistry, astronomy, physics or biology. Essentially the mission of the work-package 4 about Scientific Visualization is to support scientists in visual analysis of their data typically produced by simulations or computations on abstract mathematical models. By providing visualization methods and technologies as well as improved analysis and modeling possibilities by the partners involved in WP4, transnational researchers can make use of interactive visual representations of their data. VISIONAIR scientists completed 29 projects within WP4.
Target groups for the transnational access may include:
Scientists that want to try out their visualisation methods or systems in truly immersive environments.
Scientists that want to connect their simulation code to high-end visualisation platforms to do on-line simulations with real time assessment of the simulation results.
Researchers who need high definition, high resolution output devices they usually do not have access to.
Scientists that develop novel post-processing methods or create parallelised versions that require a large computing effort.
Scientists that want to visualise massive data sets that are usually too large for common visualisation facilities.
Guests of the transnational access programme will not only have an extensive visualisation platform at their disposal, but also further resources that are required to undertake their visualisation tasks like supercomputing resources, high performance networking environments, experimental facilities and others. Although these resources exist and are available to the programme participants to a limited degree, proposals are required to be visualisation centred and should only in rare cases draw heavily upon these.
The projects comprise visualizations of huge data sets from simulations with the need of a high degree of interaction capabilities, researchers increased insight into their data by using innovative visual analysis and modeling techniques and improve research results and outcome in various research fields. Not only using scientific visualization, but some projects also focused improvement of visualization techniques for application in education or perception and awareness of 3D content focusing scientific visualization in general. Some projects also focused improvement of visualization techniques for medical or forensic applications for instance, which will further be used in training, education and research and further improve specific applications and scientific visualization in general.
It is not the purpose to cite all the projects here but we can cite the construction of a virtual representation of Mars built with the data coming from observatory enabling by cave immersion a transportation of scientist on Mars. This project was enabled at the Thinklab in Usalford. A more conventional project which comes with very nice image production concerned the evaluation of the turbulent incrompressible flow and the particle tracking in the Axialzyklon wih a high-resolution model. The scientist were immersed in a flow of visible particles to better understand the results of simulations. The ligth sheet experience is another project were the support of immersion demonstrated a very high interest for scientist. Biologists were able to clearly navigate inside insect and to better analyse the insect structure. More rapidly and easy that never before: during the last general assembly, the guest Dr. Emmanuel Reynaud claimed that accessing an infrastructure like VISIONAIR was a strong opportunity for biologist to produce new publishable knowledge.
• Ultra High Definition Imaging distributed on optic network
The infrastructure offered by the partners in UHD-NET (work package 5) provides facilities for visualization of scientific experiments. Among the others, the following research projects have been identified as potential users of the infrastructure:
Telemedicine, e.g. experiments with streaming from hospitals (operating rooms) for remote diagnosis and support,
Radio-astronomy, e.g. experiments with streaming high resolution pictures to remote participants for analysis,
Chemistry/Biology, e.g. live streaming from experiments to remote audience,
Cultural sector and sport, e.g. experiments with real time transport and visualization of important cultural and sport events, enabling research and innovation in cultural interchange experimentation by combining culture and high-technology,
Physics, e.g. real-time and offline scientific visualization using high resolution, e.g. remote rendering of scientific processes and simulations.
Other potential services and appliances of the infrastructure could be:
Telepresence and videoconferencing in very high resolution, excellent quality and low latency (uncompressed HD streaming),
Education and e-learning, e.g. high resolution transmission of lectures and presentations to the remote audience,
Entertainment, e.g. digital cinema with on-demand video transmission from remote location, gaming and immersion in virtual reality (3D).
In recent years there was rapid development of UHD devices, so the technology became more and more popular. The technology became also much cheaper, however professional devices are still expensive and unavailable for smaller projects or scientists, especially considering streaming UHD/4K technology. Many European projects and researchers started their interest of new technology related to UHD, 4K, 8K and stereoscopy, so that work-package partners (WP5) had much bigger interest from the potential users.
The principle of User Access was udapted so that some facilities could be be moved towards the applicant when necessary.These conditions enabled new TNA projects to be conducted and completed with a new model where the host becomes the guest and carries the necessary facility to the guest.
Most of the 19 Trans-National Access projects completed in WP5 was ultra-high definition streaming experiments for various scientific appliances or scenarios, just to mention: medicine, robotics, e-learning, sport, arts and culture. WP5 is composed of 4 installations, which are located in: PSNC (Poznań, Poland), University of Bristol (Bristol, UK), I2Cat (Barcelona, Spain) and University of Twente (Enschede, Netherlands). It must be noted that initially a team settled in UESSEX was involved in VISIONAIR but a part of this team moved to Ubristol. Then Ubristol replaced Uessex in the consortium. All installations are equipped with Ultra High Definition or 3D facilities and are interconnected with high speed optical networks (GEANT, GLIF, HPDMNET or other). Installations may be used separately or combined for complex scientific experiments. All the facilities are capable of supporting very high quality distributed visualization applications. A new and unique capability is offered by the combination of advanced visualization technologies and network services. The network services enable multiple high-resolution digital-media streams to be transported among global sites, including University of Bristol, PSNC, I2CAT and University of Twente, using dynamically provisioned optical light paths across multiple domains, which can be used on a scheduled or on-demand basis. The infrastructure offered by the partners in this WP provides facilities for visualization of scientific experiments.
WP5 partners showed that they are ready for common collaboration. Techniques, methods and integration of technologies, they worked on for last 3 years, allowed for making more complex experiments at the world level. With support of WP5 partners some of them were the first demonstrations of cutting-edge technologies in the world – for example Human-robot interaction over 40G network with UHD 3D High-Frame Rate at distance 2000km with minimum latency (TNA 114).
Some projects like project n° 136 (Research on new enablers for artistic virtual environments and performances) linked both WP5 and WP6 (using UHD-NET and VR installations). It demonstrated the capacity of VISIONAIR to integrate facilities together and provided really amazing collaboration (this project has already be mentionned sooner and concerns the organisation of a concert with players and dancers spread in four different locations in Europe). The Work-package allowed also the Hypermed project which investigated the use of 3D Ultra-High definition and high quality streaming for tele-medecine application. As already mentionned, this project was awarded by the Celtic excellence program.
This work-package developed a particular model of trans-national access within VISIONAIR with very impressive results. Tele-presence was also used to support sportive gesture analysis. The sportsman or group of sportsmen is working alone but observed in 3D by a remote coach. The coach can record the session and enter in close collaboration with the sportsmen even from a very far location. This kind of environment is quite new and expects a strong technical support. In all the use-cases, the final users, sportsmen, coaches, artists gave evidence about the new opportunities offered by the corresponding environments. It was also demonstrated the importance of the scenario organization by the involvement of end-users: WP5 created a space of convergence between endusers and technology developpers.
• Virtual Reality
The Virtual Reality facilities propose immersive environments. This is a huge work-package clustering a important number of facilities. Mainly virtual reality CAVE are concerned here, but also usage of interaction systems as haptic devices. The immersion capacity is also enable by Head Mounted devices and involve high level tracking solutions. The specific case of the GRIMAGE cave in INRIA grenoble creates a scanning solutions and has the capacity to produce in real time a 3D avatar of the observed scene. Many CAVEs were proposed by VISIONAIR including two huge equipments with the RWTH in Aachen and IMMERSIA in INRIA Rennes. Both are non conventional equipments by their size.
VISIONAIR hosted on its VR facilities, a large spectrum of projects centered on new usage of Virtual Reality. 51 projects were completed there. The projects are widely distributed along an axis going from basic, fundamental research on human behavior, in its sensorial, motor and cognitive aspects to the development of new interfaces and declination of Virtual Reality with the motivation to welcome young scientists wanting to test new ideas and push forward the state-of-the-art of Virtual Reality.
On a formal point of view, the outcome of the Work package were in line with the objectives, which was a hard constraint for young facilities (e.g. the engagement of making 20% of its full running time available to other scientists and engineers within Europe and associated countries). However, this also constituted a strong motivation for the platform to develop their activities and the residency of European researchers was a clear benefit for their visibility.
Research activities focused mainly on human behavior in immersive environments, including distant collaboration, ergonomics, robotics and neuroscientific investigations. To select a representative panel of project among 51 actions is hard but let enumerate a few ones:
Sports-men gesture analysis: The test of basket-ball gestures in Rennes or rugby gestures in CRVM measured the gap between the gesture in a virtual environment and the gesture in real space. This was obviously challenging because it was necessary to shot real balls in the CAVE which is a fragile environment and to measure the initial movements to integrate it as initial condition for virtual simulations. It opens indoor training protocols.
Cognitive and psychology studies: perception of the virtual environment is a classical issue in virtual reality which remains a topic of interest. The new step here is that the facilities were expected to provide efficient results for the end-users and thus it was necessary to over-pass virtual reality intrinsic difficulties. Feedback control experiences with very high quality haptic devices were conducted at Poznam University of Technology. In CRVM again Virtual Reality was used to measure attention when driving cars, etc
3D sound experiments: several projects expeted to explore the capacity to immerse persons in 3D sound environement. Some partners invested in new devices to provide this resoure and first test were driven. What is the effect of a 3D sound on the user focuss ? What is the quality of perception of sound directions ? It was measure the threshold of sound perception by a person who perceive a sound without visual feedback and the trick introduced by a fake visual feedback. Such experiences create clear knowledge about the sense hierachy.
Virtual patrimony: the 3D scanner capacities were used in particular with ECN to advance in building virtual copy of old mockup of Liège. New developments were supported to create a complete information system storing all the references of such historical data and innovative virtual environments were produced to be integrated in museums.
Experimentations of co-located activities between several CAVES were organized. One usual challenge is that CAVEs are rarely controlled with a standard software. Moreover the complexity of contents and interactions makes the remote interoperability hard to enable. Sztaky provides a co-space solution named VIRCA, where we can share with standard internet connections a virtual space. Many successful experiences where done to associate CAVES into a VIRCA space: as an example UNIKL connected its CAVE and collaborated with SZTAKI to share a layout organization of a plant with persons in SZTAKI. This capacity was demonstrated in various locations. Other experiences enabled to connect CAVEs by sharing softwares or trying to provide other dedicated connexions.
Obviously all the previous examples do not cover whatever was done by VISIONAIR but they highlight how VISIONAIR opened a wide variety of new use cases in Virtual Reality. We are convinced that it is the key challenge for the renewal of this discipline and the corresponding technology.
• Augmented Collaborative environments including holography
The ACE (Augmented Collaborative Environments) infrastructure (WP7) is built around the need for researchers, to combine their expertise and work power in order to take up contemporary challenges: sustainable development, climate changes, healthcare,novel product development processes.... Great scientific breakthroughs are often invoked in teamwork, and the complexity of scientific problems call for a deep collaboration between scientists.
This observation is true for research inside well defined communities but also for interdisciplinary work. Distinct scientific communities will to fosters their knowledge and competencies on common artefacts that do not belong to one sole scientific field but cross the frontiers of disciplinary work. This work package aims at providing collaborative environments for researchers of various communities, as we claim that direct human interaction are as much valuable as mediated interaction. Visualization must be considered as a key point for mutual understanding and ability to share scientific data and representations, in a synchronous manner.
The equipment proposed in the infrastructure allows multiple co-located persons to visualize and exchange on the same perception (Holographic displays and tables, stereoscopic 3D High Definition projections...) or on multi-form representations. Further more, Augmented Reality (AR) facilities ease the use of mixed visualisation with any possible combinations of real (physical) models and/or digital models. As a result, expert's personal representations of knowledge can be superimposed to form and share new knowledge patterns; Technologies vary from one installation to another, depending of the requirements of users for seamless interactions.
However, visualisation is not enough to support collaboration. Consequently, each installation help concurrent interaction with these numerical representations, thanks to haptic devices, multitouch surfaces, interactive boards or innovative interfaces.
Collaboration may also be provided for distant sites. Hence, interoperability between installations is made possible so that costs, times and environmental impacts of travelling may be drastically reduced, providing that new usage spread of the users community. Video conference systems, available on every installations, reinforce the capacity to interact between distant researchers, sharing vision and applications.
Finally, many facilities can contribute to the materialization of the researchers activities: concept or objects may be prototyped via rapid prototyping facilities, and scientific argues, decision making processes and knowledge expressions can be captured and recorded on long lasting support (audio and video recording facilities). All these equipments rely on (or involve) commercial and lab-made softwares which enhanced the entirely visual collaboration.
Services provided by the infrastructure partners are currently rather engineering sciences oriented, due to the scientific background of the partners. As a consequence candidates researchers to a transnational access will experience cutting edge technologies in the field of product design, virtual manufacturing or decision making process. However, multiple experiences have proven that such devices could serve a wider range of scientific communities.
TNA projects were completed on the ACE platforms. It proves that the collaborative and augmented reality facilities of the infrastructure have achieved a good level of recognition in the European scientific community. New topics were studied extending the range of competencies of the host teams. The new offer provided by the new OPSIMLAB facilities (CRANFIELD) focused on project built around maintenance issues (simulation, training...). Arising during the last year this new facility demonstrated its efficiency and attractiveness with 3 projects.
Here also a wide spectrum of disciplines were welcomed: researchers from ICT, Humanities, Social sciences, Life sciences and biotechnology, mathematics and material sciences. The themes of the projects were also very various, with interests in ergonomic, cultural heritage, museum applications, 3D sketching, healthcare “war rooms”.... Once again, technologies were adapted to suit the needs of the researchers coming from Malta, Italia, Republic of Ireland, Lithuania, Poland...
It's worth to be noticed that partners have joined their forces to allow distant interactions such as in project 156. Such collaboration occurred also in other WPs, illustrating that the collaborative dimension went over the organisational edges of the work-packages.
To highligth a few results among the 21 projects :
The last supper interactive project was supported by Grenoble-INP to integrate 3D haptic device in a story telling application for a new museum presentation of the last supper painted in Milano by Leonardo Da Vinci. Haptic device was successfully connected to a virtual representation of the room where the paint is settled opening new interaction opportunity to better discover this fabulous painting.
Experience to enhance the capacity to better understand 2D technical drawing by projecting a 3D view over the 2D drawings were conducted in Milano. This clear augmented reality demonstration opened new trends for learning or creating new CAD models for industrials practises.
New collaborative practises where developed and experimented to schedule the nurse plan for home care organization. Currently, in the region of Lecco, this work is done by a team of planners who just discuss about the best opportunities when a random event occurs. The introduction of multi-touch table demonstrated its efficiency to enhance tangibility and capacity of collaboration. In the same kind of direction the Design of a Healthcare Analytics War Room to support healthcare data visualization for better problem solving and decision making was supported by University of Twente. This projects highlights the demand of more tangibile and intuitive management of potential crisis scenario.
A more classical application within the scope of augmented reality is the Design of a Virtual Reality Environment for MAintainability TEsts and MAnufacturing Systems Simulations supported by Ucranfield. The potential applications in production systems are arising and the demand to explore the potential deployment of the corresponding technologies is increasing. Here the challenge is to provide mature enough environment to be used in the conditions of a production system.
It was also explored new interaction systems to create 3D parts and to support direct 3D modelling for artists and designers. The results of studies conducted in IPK environments demonstrated the feasibility of the corresponding processes.
Results from joint Research activities
The joint research activities were mainly conducted to improve the quality of service of VISIONAIR respect to TNA projects. Split into three work-packages and more deeply into tasks the organisation of the plan has been reorganised around 20 actions at the end of the first year as anticipated in the description of work.
• WP9
Improvement of human-VR interaction is a key point of many currently performed research projects. Within the VISIONAIR project we would like to conduct our research in this area and what is especially important, we would like to invite researchers from allover the word to visit our laboratories and to make common research. By this way, we will obtain interesting results, establish and improve our cooperation with other partners, which will enable us to form interesting topics for future research.
This work package is concerned with advancing the core methodology to improve the possibility of interaction and collaboration between people through the services to be provided within VISIONAIR.
This will be possible only if we define
1. How we can add interaction capabilities to digital models;
2. How we can improve the possibility of interaction and collaboration on these models;
3. How we can let the distant systems collaborate;
4. How we can propose new methods for new interfaces as holodisplays.
The first point (1) consists in defining new methods for preparing the virtual scene independently of what it is built. This virtual scene preparation needs more and more focus on digital mock-ups that are the core of development of virtual products. This will clearly improve our capacity of access, especially with respect to the WP6 (TNA, Virtual reality) but the developed method should be help toward scientific visualization in our environments (WP4, TNA). The second point (2) aims at allowing effective collaboration on these virtual environments by improving immersion. The proposed methods will be useful to improve usability of our environments on place (WP6, TNA virtual reality) but also between equipment (TNA, WP7, Collaborative environments including holography). The third point (3) aims at building the conditions that allow an effective collaboration by sharing the virtual scene between access points and is clearly dedicated to WP7.
The following actions were conducted in the scope of WP9:
Action 9.1. Integration of multimodal data and kinematic models into VR simulations (GRENOBLE-INP, INRIA, AMPT, CNR, UPATRAS, ECN).
VR and AR facilities can potentially improve product development process when they applied to assembly/disassembly (A/D) simulations, support for assembly and maintenance, ergonomic studies, virtual prototyping in the context of conceptual design and product evaluation. Despite the AR/VR devices’ price becomes accessible for small and medium companies; these technologies are currently rarely used because of the lack of tools to prepare the digital mock-up data for these VR and AR applications. Indeed, the setup of VR simulations requires the transformation and composition of many types of data: the geometric description of the product and its environment, engineering instructions for A/D and maintenance, ergonomic postures, multimodal data for product representation. While these data are often pre-existing, they must be transformed, enriched, and combined (geometric simplifications, combinations of 2D and 3D, …) to enable their use in VR/AR applications. This JRA activity proposed methods and tools that improve the link between multimodal data and VR/AR simulations. Engineering data (CAD, PLM, assembly instructions), as well as multimedia data (images, meshes, posture capturing) are processed with tools and methods to facilitate high-end VR/AR applications.
Action 9.2 Virtual Factory service (CNR, UNIKL, AMPT, UPATRAS, ECN, POLIMI, KTH)
In the industrial manufacturing field, Virtual Reality environments offer good facilities for the assessment of product design solutions according to various points of view (e.g. productivity, working conditions, safety and ergonomics). This is even more important for complex products such as a complete factory. Unfortunately, some limitations exist that prevent the full exploitation of Virtual Reality environments by engineers. Despite the amount of knowledge and know-how contained inside datasets of tools like CADs and PLMs, it is difficult to find the good rule for, on a side, extract the data for building virtual environments and, on the other side, for gathering the digital models that fulfill certain functionalities. PLM/CAD systems provide detailed specification of the conceived product solution, which does not satisfy requirements of VR environment, both in terms of type and complexity of representation. Therefore, these data need to be prepared and adapted to the VR applications. Understanding the exact requirements to fulfil and the needed data transformation and processing sequence requires specific skill and experience that somehow limits a wider adoption of VR facilities to a larger audience.
Research carried out in Action2 was aimed at designing and developing digital
services helping users, on one side, in the transformation and processing
of data from their original CAD/PLM world into representations suitable for
Virtual Reality simulations and, on the other side, on collecting and retrieving
digital models arranged with a specific industrial and logical classification. Focus is on the classification and archival of models for the digital factory design.
The results are integrated in the VISIONAIR VVS (Virtual Visualisation Service) in the form of a manufacturing view of the shape repository and in a repository of workflows.
Action 9.3 Integration and evaluation of interaction potential of Handheld Devices into VR systems (UNIKL, INRIA, UNIVMED, GRENOBLE-INP, UCL, USTUTT, SZTAKI, CNR, AMPT, IPK)
The goal of this Action was to analyse and assess the supportive potential of different interaction techniques for the interaction within virtual environments. One focus was the consideration of Handheld Devices (like smartphones and Tablet-PCs) as interaction device within immersive virtual environments. Therefore multiple prototype applications have been developed to analyse the fundamental applicability and the supportive potential of Handheld Devices as mean of interaction in virtual immersive environments.
As a second focus, investigations have been made how an initial user need
can be supported and satisfied by proper interaction techniques in the best way. The research question was: How to identify interaction techniques which fit to certain user purposes in an ideal way? Therefore interaction techniques need to be described in a non-technical way, outlining the functionality they provide and the benefits which can be achieved for certain applications. By conducting such research, even non-VR experts have the opportunity to understand the characteristics of sophisticated interaction techniques and compare them in the light of well-known applications, from end-user perspective.
For the design and implementation of the new interaction device the research of Action 9.3 and 9.9 was combined for this activity. Within Action 9.9 design and development was being focused to design and develop a device class, which is inexpensive and easy to adapt and extend to different user requirements. The research question addressed here was the integration of the device into virtual environments as well as the evaluation of the design by users in a CAVE. The assessment consists of various applications with different level of interaction. Out of this assessment various improvements and features have been added to the design.
Action 9.4 Annotation capacities for asynchronous collaboration (RWTH, INRIA, UNIVMED, ECN, CNR)
For the creation of an annotation system that facilitates the asynchronous collaboration in VR environments, the partners involved in this joint research activity identified three main areas that need to be addressed: storage & retrieval of annotation data, creation of annotations, and presentation of annotation data. The development of a comprehensive annotation system that covers all three aspects is a challenging task that goes beyond what can be achieved from within a VISIONAIR JRA only. As such, we (partner RWTH) have combined work from multiple research projects funded at other levels, making efforts to develop solutions for each area. Within VISIONAIR a prototypical annotation system based on this work is now available.
ECN has been working on reverse-engineering, 3D visualization and knowledge management for cultural heritage. We started a project in 2008 with a museum to apply theoretical research on museum collection objects. In 2011, we started a collaboration with Belgian researchers to confront our ideas and methodologies. We published two articles in order to set up a new methodology for cultural heritage objects capitalization. It goes through 3D digitization, information modeling and visualization.
Action 9.5 Real time 3D Modeling for Interaction and Presence (INRIA, UPATRAS, UCL)
3D modelling sensors relying on cameras sometimes extended with depth sensors enable to compute in real time a 3D clone of the user. The popularity of such approaches is growing rapidly, the Microsoft Kinect probably be the most visible example, this device enabling marker-less interaction in games. In VR environments the full potential of such devices remains to be explored. The geometry extracted opens the way to marker-less interactions, while, once complemented with photometric data, it enables to build a virtual clone of the user usable for enforcing user’s presence in the virtual environment. The goals of this action is to explore the potential of real-time 3D modelling for improving the user’s ability to interact with the VR environment, to collaborate locally and to enforce its telepresence. In addition, the potential for use of such interaction devices in holographic applications, will be explored and new interaction metaphors will be developed for facilitating local collaboration and human-centred interaction.
Action 9.6 Action to awareness in 3D environments (SZTAKI, USTUTT)
Awareness provides a way to get informed or understand the situation regarding entities of interest. Awareness information generation deals with deciding what context and raw information is necessary to create awareness of a specific situation or state. For example, to generate data that describe the fact that a specific user is “actively working“ requires not only the “user presence” context information but also historical data about what he has done. One important research topic in this area is to figure out what information and in what details is necessary or useful to be presented for the users. The importance of awareness information increases
in the case of remote collaboration, where the better understanding of the remote partners’ status might be essential for the efficient collaboration.
MTA SZTAKI proposes a generic awareness handling system suitable for Virtual environments. This proposed system consists of a central message repository
and distribution server, several data collector agents and several data
consumer agents. The system is suitable for adding and visualizing
contextual data in a given collaboration environment.
Virtual reality installations usually are custom designed to fit the target application based on experience of users or companies. The design and capability essentially affects usability, a CAVE might not be used as an static image viewer or a 3D Desktop VR on a small display might not gain full user immersion for instance. From this point of view, to assess and to evaluate hardware and projection setup design corresponding target applications a set of criteria for grading utilization and interaction of the user is needed and has been proposed within this research action.
Various tracking data from virtual environments regarding user interaction in specific applications or application classes respectively is being harvested and stored in a data base for qualitative comparison between hardware setups running different applications. This will improve future design matching specific applications to potentially increase immersion and awareness in 3D environments.
Action 9.7 First versus third person perspective for large scale navigation (UNIVMED, AMPT)
During the project, we conducted a number of experimental studies, in which we manipulated a number of factors, while measuring cybersickness symptoms, using the standard "simulator sickness questionnaire" (SSQ, Kennedy et al., 1993). Participants were immersed inside a 4-sided cave system, and submitted to a spatial retrieval task, in which they had to find a place (goal) in a large complex environment.
In a first experiment, using first-person navigation with a flystick, we tested the effect of sensorial aids, a spatialized sound or guiding arrows on the ground, attracting the user toward the goal of the navigation task. Results revealed that sensorial aids tended to impact negatively spatial learning. Moreover, subjects reported significant levels of cybersickness. In a second experiment, we tested whether such
negative effects could be due to poorly controlled rotational motion during simulated self-motion. Subjects used a gamepad, in which rotational and translational displacements were independently controlled by two joysticks. Furthermore,
we tested first versus third-person (avatar) navigation conditions. In the
second condition, cybersickness tended to be lower. However, large
interindividual differences made the difference non significant.
A third experiment was realized, in which we simplified the environment. The participants simply had to reach the exit of a one-way maze. To do so, they were given three locomotion interfaces, across three different test sessions. The first interface was an Xbox controller, on which two minijoysticks independently controlled forward motion and horizontal rotation. The other two interfaces consisted in custom adaptations of a "redirected walking" algorithm: users had to physically turn their body toward the direction they wanted to go and a slow counter rotation was then applied to the virtual environment, such that they were redirected to face the cave's front wall. The general hypothesis was that this would prevent fast visual rotations, hence potentially reducing cybersickness. In the second interface, users had to walk in place, and passive captors on their knees were used to calculate locomotion (forward) speed. In the third interface, the Xbox controller was used for forward speed along with the "redirected" algorithm. The comparison between these last two interfaces was meant to measure if active stepping would reduce cybersickness. Results from this study show that 1) redirected walking is easily accepted and intuitively used by all participants; 2) that walking in place is reported as tiring and results in greater speed variability than the other two interfaces; 3) that, on average, all interfaces result in self-reports of cybersickness, even if exposure time was quite short, always under 10 minutes. Finally, it appears that users having no experience playing video games exhibit higher scores of cybersickness than "gamers", suggesting that, besides interface properties, individuals factors come into play.
Another study related to navigation investigated the use of natural language and user gesture for navigating in real scale virtual mock-up of a building. Two sets of interfaces were introduced: 1) interaction devices, 2) natural language (speech processing) and user gesture. The survey on this system using subjective evaluation (Simulator Sickness Questionnaire, SSQ) and objective measurements (Center of Gravity, COG) showed that using natural language and gesture-based interfaces induced less cyber-sickness, as compared to device-based interfaces. Therefore, these results suggest that gesture-based interfaces might be more efficient than device-based interfaces.
Finally, a study related to driving simulation has been conducted. The added value of vibrations during driving simulation was studied. Current dynamic driving simulators induce simulator sickness and it is still difficult for the driver to project himself in the virtual reality, due to incoherent multi-sensory stimulation and perception. To know the effect of vibrations on a subject, the effect of a whole-body vibration must be defined, as the sources of vibration in a car cabin. After determining all the parameters, a formula was proposed, to produce vibrations as a function of the car state, the road and the boundary conditions. An experimental study was realized, with nine subjects. In order to do this experimentation, three actuators were installed inside the cabin of the car driving simulator. Data analysis is ongoing.
Action 9.8 Deployment and analysis of collaborative software (INRIA, UCL, UNIKL, SZTAKI)
The Joint-Research-Activity deals with the implementation of VR distance collaboration tools and their usage in the scope of multiple application fields. The Research Question was: How can distance collaboration tools be deployed to different VR systems and beneficially applied for different research topics?
Thereby the development of new systems that allow the coupling of different immersive VR systems was not the objective. The goal was to test out the capabilities of existing distance collaboration tools (in detail COLLAVIZ and VirCa) and assess them against each other, based on certain use-cases. The first topic of interest was how complex the deployment and adaptation of academic software prototypes on new VR systems is. Challenges in the installation and usage of software prototypes to Partners infrastructure who have not been involved into the development of the tools itself were analyzed. The second topic of interest was if the analyzed distance collaboration tools can completely support the dislocated work for certain use cases. Thereby interaction, visualization and collaboration functionalities are crucial. Therefore two typical use cases (collaborative visualization of seism data and a factory planning project) have been developed and executed by the help of the software tools.
Out of this assessment different distance collaboration tool features have been identified and assigned to typical use-case demands. Criteria to improve the future development have been proposed, as well as advices for the usage of the distance collaboration tools could been given.
Action 9.9 Interactive devices and tangible interfaces (PUT, IPK, UNIKL, UNIVMED, USTUTT, SZTAKI).
The applied today system for human communication to VR systems include only: different human tracking systems, joysticks and haptic devices. In VR systems there is still a need for development of new human-VR environment communication methods. The aim of this action was to support the development of highly interactive physically enriched VR systems and to ease the integration of various types of interaction devices, physical (tangible) objects and tracking technologies.
Furthermore a haptic manipulator (haptic joystick) was developed by PUT and integrated into an stereoscopic projection system and a tracking system in order to study the integration of various kinds of interaction devices, physical objects and tracking technologies within the VISIONAIR JRA and TNA projects. The built haptic device (light robot) enabled the improvement of the human - immersive VR communication and interaction. The main task of the haptic device was to move its ending arm to the position in which the operator's hand may touch the visualized object. The haptic device expanded the immersive VR system with tactile feedback.
In the project, the Fraunhofer IPK developed a software framework called Tangible User Interface (TUI) framework which provides a high-level interface and duplex access to important properties of interaction devices, interactive objects, and interaction tools.
The action was also focused on enabling haptic feedback in immersive case scenarios. To do so, a new haptic device was constructed and an appropriate VR software was developed at PUT.
Furthermore, after deep analysis it was decided to integrate the newly developed by PUT haptic device with the TUI framework worked by IPK. Initial tests proved that such an integration was possible by using the UDP network data exchange protocol.
Action 9.10 Research into Cognitive Infocommunications (SZTAKI)
CogInfoCom proposes a new and unified conceptual approach in which the process of merging is derived from the theoretically unified concept of different levels of cognitive capabilities co-existing in the information space (irrespective of whether they are natural or artificial capabilities, and whether they are individual capabilities or capabilities which emerge from a cloud of artificial and/or biological components). This derivation extends to various aspects of the merging process, such as interactions and communication, as well as increasingly flexible interfacing between networks of living beings and artificial cognitive systems etc
SZTAKI is managing the CogInfocom conference but many partners (Grenoble-INP, INRIA, UNIKL, etc) were participating to the action with the organisation of special session centred around the VISIONAIR activities.
• Advanced Technology for Interaction and Collaboration (WP10)
The following actions have been listed in accordance of the tasks described in the Dow:
Task 10.1: Holography technology improvements and technology evaluation of high resolution curved tiled display
In this task the holographic technique used in VISIONAIR is improved. The work has been concentrated on developing the technique so it can be used in a collaborative environment - both locally, with multiple users at one display, and remotely, connected to other displays which may be of another kind.
The Holotable is an autostereoscopic display created at KTH. Its previous prototypes could only be used by one person at a time and had, hence, limited usage as a collaboration tool. It has now been developed to a multiuser display in the form a table where two users sit on opposite sides and can see a 3D image floating above the table surface. In the previous prototypes one linear array of projectors provides a viewing zone for one person or possibly several persons sitting side by side. For the new version the design idea was to add one more array of projectors, oriented parallel and displaced perpendicularly to the first array, to provide additional viewing zones on the opposite side of a table-top display. These additional viewing zones can show images independently of the first viewing zone, e.g. the opposite side of a three-dimensional scene. The HOE has to be modified somehow so it can show the viewing zones from one array of projectors on one side and viewing zones from the other array on the opposite side without mixing the two. We have investigated three methods for achieving this. We have also modified the software accordingly. The current prototype uses an HOE of size 30 x 40 cm. Our single-sided display has a 60 x 80 cm display area, and we expect that we can make a double-sided display in that format as well. We use four LED-projectors on each side. The projectors are driven by two ATI Radeon HD7970 graphics cards mounted in the same computer. A maximum of six projectors/screens can be connected to each graphics card. The image quality is virtually the same as for the single-sided prototype. The view-zone size and viewing angle for each viewer can be the same as for a single-sided display. We have also shown that it is possible to make three and four-sided holotables.
Task 10.2 Create an Interoperability Environment among Distributed Platforms
This task concentrates on network transport and protocols, streaming and codec technologies and on developing comprehensive vision for WP10 work scenarios. The scenarios form the base for evaluating appropriate state of the art technologies regarding networked communication, collaboration and interaction. They also define the demands on contenIn this action the work was focused on software development and establishing network environment for remote collaboration in order to enable the scenario defined as a research question. To achieve this goal it was necessary to fulfil several requirements. Significant part of work has taken adapting VITRALL, a distributed web based visualization system enabling a real-time visualization of a complex 3D content using remote distributed servers equipped with modern multi-GPU solutions, to support functionality required for proposed scenario. It included: 1. Integration with JPEG2000 streaming platform used by two VISIONAIR partners: University of Essex (now the team in Bristol) and PSNC as well as external users, such as SURFNet and WAAG Society from Netherlands, CRC in Canada or ITAI PAS in Poland, 2. Implementation of new module supporting strict requirements of video streams synchronization for stereoscopic and holographic displays, 3. Implementation of new JPEG client in order to support multistream displays - UHD 3D in PSNC and Holographic table in KTH, 4. Enhancement by functionality of reading remotely located 3D models, to enable sharing of models between rendering and modelling systems. 5. Implementation of new module for tracking and uploading modification done to the model by external software – to allow integration of the system with arbitrary modelling software (Blender, 3ds Max, etc.) Platform integration activities focused also on the network streaming solutions, advanced streaming technologies required for transmission of ultra-high-resolution multimedia streams has been tested. The main aim of network experiments was to verify integration of different platforms and identify possible issues. It was also necessary to check the throughput and latency to assure high quality and comfort of on-line cooperation of designers. Tests were conducted gradually to validate all elements of the system and eliminate limiting factors. Additionally, HPDMNet (over GLIF physical infrastructure) and GEANT connections (direct or through OpenVPN) were established in order to achieve research goals of this action. Finally after fulfilling all conditions necessary to successfully realise proposed scenario, it was shown on VISIONAIR Open Forum 2014 in Poznan. The demonstration has shown remote collaboration in modification, rendering and visualization of 3D models. In this performance was used the scenario, where a group of people can work on one 3D model and perceive the same virtual object from different perspectives and views using different kind of displays such as holographic table or UHD 3D displays. The scenario presented during the Open Forum conference involved designing a children’s playground. In this process there were three sides located in different places and with different visualization capabilities. Each of them had the opportunity to view created model independently, discuss on-line all changes and make decisions about further designing actions. One of the parties involved into demonstration was a 3D model designer located in Barcelona. He used Blender modelling software for editing 3D model and making changes proposed by the other participants. 3D models were shared across the network, so all participants were able to see effects of his work immediately. The second person was sitting in front of holographic table in Stockholm. VITRALL server located in Poznan was rendering 3 different views for 3 projectors serving the holographic display. Holographic table operator could move and rotate the 3D model in order to see it from all sides. The third side was an audience on Open Forum in Poznan. On 4K 3D screen there were projected all views of models as well as two remote sides via videoconference. Demonstration allowed presenting in an accessible way capabilities of VITRALL. In this live design and discussion process the playground project was being modified by inserting additional items (swings, ladders, etc.) and changing their position and appearance. The show has generated a lot of attention, as easily applicable for real-world scenarios, arousing many questions after it.
Task 10.3: Create technologies for collaborative communication and interaction between networked holographic table displays
The partners will collect information about, implement and evaluates interaction technologies appropriate for holographic table autostereoscopic displays and similar displays. As there are very few holographic tables around there is a need to develop new interaction schemes and tailor requirements in accordance with developed application scenarios.
The work focused on the development of a solution which offers the possibility to have live remote view of virtual and dynamic objects. These objects can be a combination between physically present objects and virtual objects, and should be live viewed from one or more remote locations. To experience and research the possibilities within this focus multiple project have been executed and tested.
VITRALL
NICE
Distance collaboration support environment
3D printed head mounted devices for communication and discussion
Remote lab management
Virtual Factory
All solutions allow for viewing objects from one or multiple locations. The differences can be found in the number of participants, the combination between real and virtual objects, sharing digital environments and controlling remote locations.
The work described in this document focused on enabling and experiencing live remote view of virtual and dynamic objects. The interaction between viewing real and virtual objects, in combination with interaction possibilities and ability to do this over a large distance, was examined and tested in multiple settings.
• Conceptual Modeling, Assessment, Metaphors and Social Impact (WP11)
During Period 3, the main activity of WP11 was to finalize the actions which have been defined at the end of first year within deliverable “Visionair-Deliverable 11.7” : Detailed Research Program - Conceptual Modelling. This deliverable was reorganising the tasks of the Description of Work into six main actions. During period 3, the technical work was finalized and reported on the VISIONAIR site as for WP9 and WP11. WP11 was concerned by the following six actions:
Action A11.1: Modeling the user engagement with VISIONAIR: An Object Process Modelling representation of VISIONAIR TNA process was constructed and mapped in a 3D space
The action consisted of two steps corresponding to the two research questions mentioned above.
Modeling of TNA process using the OPM framework: A team of students from Technion was in charge to translate an oral definition of the process presented by the VISIONAIR scientific coordinator into the OPM framework. We designed a detailed conceptual model of the TNA process. This task presented an opportunity to deeply analyse the TNA process and get a better overall system-wide comprehension of the TNA process sub-tasks.
Instance representations: In this original work we visualized the system after observing that even though the conceptual model is quite clear, the visualisation of more than 200 projects make the perception of instances very hard. To alleviate this problem, the conceptual model was converted into 3D representation that provides for direct and tangible perception of projects.
The main proposition is to develop a model that present instantiation of OPM. We tested this approach on the instantiation of the TNAs OPM model. This model was created during the first JRA year of VISIONAIR in collaboration between Technion and Grenoble-INP. The OPM model is basically a 2D graph. The main idea is to use the third dimension to visualize instances. The OPM models has the ability to represent on the same graph both the processes and the objects handled by the processes with their various states which makes a clear understanding of the potential evolution of the objects.
The model was slightly updated and reproduced in a 3D environment. Project, Review Committee, Host Facility and Research Group are the four main objects participating to the main process. In 2D the arrows clearly highlight the activities that can change an object state into another state. But if we create an instantiation diagram then the corresponding graph includes the 200 projects, all the review committee members, the 29 VISIONAIR host facilities and about 200 external research groups which submit projects. This graph will not be readable.
Thus OPM is perfect to get an abstract view of the overall process, but it does not provide a correct solution for instance visualisation. We extended the view by extruding the OPM 2D graph in the third dimension by placing in front of every object class its corresponding instances. The system was also enriched with interaction capabilities to support selection of objects and states to be visualized.We tested this new representation of the instance model as follows:
on an holographic screen we found that the resolution was too low to enable reading the names of the instances;
On a HD stereoscopic display we found that 3D perception enables a very good perception and navigation (Figure 4);
On a 6K 2D display (3x3 HD display matrix) we found that the quality of character strings is much higher and is a real improvement, but being 2D it does not provide the added value of 3D perception.
In conclusion, this 3D representation may be a real added value for huge data set perception. As soon as text is visualized in ultra-high resolution we can expect to be able to read this texts. Sterescopy is a clear advantage to perceive relative positions in the 3D scenes. The use of such representation should be adequate with UHD 3D stereoscopy displays.
Action A11.2: User requirements and expectations capture – A roadmap for constructing Synthetic Environments in VISIONAIR.
A TNA can only be efficient and effective for both parties if it is preceded by adequate preparation. Aligning requirements and assessing resource usage
and working methods are essential here. Adequacy of these preparatory processes is not obvious, as the stakeholders are geographically scattered all over Europe and have their own backgrounds and interests. Therefore, supporting the underlying deliberations can yield huge benefits in preparing a TNA.
A TNA is not a unique phenomenon. Many situations exist in which stakeholders from different backgrounds collaborate to achieve a specified and predefined conjoint goal. In product development, so called synthetic environments (SEs) are an example of a setting in which such cooperation takes place. Given the way in which SEs are developed, established, used and evaluated, there is clear similarity between these SEs and TNA's. Consequently, the preparatory processes of SEs and TNAs are also similar, and the tools and working methods used in preparing and running SEs apply to TNAs as well.
Action A11.3: Mapping user requirements to VISIONAIR resources through enhancement of the resource e-map: A tool was developed in order to suggest the most appropriate combination of the VISIONAIR infrastructure services that could best fulfil the user requirements and expectations.
This action aimed at providing a tool for the external user enabling them to assess and evaluate, through the web-based E-map tool, what facility would satisfy his/her requirements and to what extent. The requirements on visualisation, interaction, associate and software facility resulted from the E-map ontology were used for the extraction of information such as display and tracking devices, manufacturing machines and software tools, respectively. For each user type, a different route was identified for capturing the necessary user input, so as for the system that follows to ultimately provide the infrastructure classification based on that input. The expert users are able to identify more easily the specific requirements. For example, an expert user might be interested in having force or tactile feedback, while a novice user with the same requirements is questioned whether advanced hand motion operations are in order.
The algorithm which is used for the assessment of the infrastructures based on the user's requirements is by ranking the available alternatives. The alternative with the highest utility is defined as the best alternative. The utility of an alternative is calculated by normalizing its requirement values. In the current phase, all the criteria are weighted equally. The decision making for the available alternatives can be formalized in a decision matrix, where the rows represent the alternatives and the columns represent the criteria/user requirements. By using the appropriate equations the infrastructures ranking is produced.
Action A11.4: Framework for modeling and assessing collaboration process within VISIONAIR: A questionnaire provided a qualitative map of the interactions between collaborators.
This action aimed to identify the enablers and barriers that will improve or hinder collaboration activities throughout the life cycle of the VISIONAIR reviewing process. The methodology is comprised of generating a list of collaboration enablers and brakes/barriers derived from previous work and literature review. We propose a four-step process to ensure collaboration assessment:
A questionnaire is first administered to a small number of people involved in the review process to extract the most important factors which can be used to describe collaboration activity and identify the main enablers and brakes for collaboration. This questionnaire needs some training to be filled. It is thus dedicated to a limited set of representative persons and projects: filling the questionnaire, a selection of the Patel's keywords must be done. By analysing the results it was possible to identify which keywords were used and which have no sense for the given context.
A second questionnaire is administered using the most important factors identified in step 1. This questionnaire was direct and easy to answer for any person involved in the assessed collaboration. It lead to the evaluation of a restricted list of factors. This questionnaire is dedicated to be widely filled. It will ask for each phase of the assessed process to define which collaborations were used and the main enablers and barriers within this collaboration.
Analysis of results: the second questionnaire indeed provided a qualitative map of the interactions between collaborators. Then some social network assessment techniques may be used to interpret collaboration capabilities of the organisation or system which is under focus. To validate this method and adjust the questionnaire we checked it first on a well known collaborative process. It has been a good way to finalize the process.
The second questionnaire was tested on a single TNA project and enabled producing a collaboration graph where the connections between roles are highlithed and where we can see the positive and brake factors with respect to collaboration. The main conclusion is that such a method is useful and we proved it can be applied, but it needs to be refined. The information acquisition should be made more automated so collaborators can tell for every collaboration the impediments and facilitators of the collaborative work in a simple way.
Action A11.5: Conceptual Model for four visualization applications: In this JRA, we used physically small but complex system to demonstrate how OPM's Object-Process Methodology (ISO 19450 Standard) enables us to effectively communicate complex business models.
The aim of the project was to develop affordable servitisation business models based on the GE Vscan ultrasound for use in primary care. Being able to use the Vscan in this sector would permit to improve the patient experience and reduce costs for the protagonists of the primary care. Several objectives were set for this project:
Review current processes for image capture between primary care and hospitals;
Develop a series of visualized servitization models;
Develop cost/benefit analysis for the range of possible business and service models;
Validate these models against current processes with end-users; and
Explore the application of potential visualization technologies.
First, product service system (PSS) models were developed in order to demonstrate the patient's journey. Experts validated the proposed business models. For visualizing, understanding, and communicating these business models, visualisations were developed using OPM models and Vivid OPM. The first one permits to reduce the journey of the patient to one visit to the general practitioner. The second model validated the business model, based on the services provided by a radiographer who visits the surgery on a weekly basis. The third business model is an extension of the second approach. However it requires a diagnostic radiographer qualified enough to analyse the images. Finally, in the fourth model, a mobile radiographer visits the patients at home. All four models were validated to be feasible, so a healthcare organization can pick what best fits its policy service model. Quantifying the cost of each model will enable finer optimisation of the model to be selected. In addition to its value for promoting affordable healthcare services, this JRA serves as a demonstrator of how OPM Object-Process Methodology (ISO 19450 Standard) can be utilised to effectively evaluate visualization technologies in general and those developed under VSIONAIR in particular. In this project, VISIONAIR has made notable contributions to healthcare in the public sector.
The wide array of research domains and applications related to visualization is a clear demonstration of the success of VISIONAIR, to be a focus of knowledge in all aspects of visualization across Europe and a world-class center for visualisation technologies and research. VISIONAIR is continuing its journey in advancing the state-of-the art in visualization and dissemination of visualization-related knowledge to all the interested European researchers.
Action A11.6: Model based approach for developing collaborative environments: This work presented (1) ergonomic issues (2) awareness issues- (3) collaborative environment for real world applications.
We aimed at developing a knowledge base that can be used to understand the complexity in developing collaborative environments for multi-functional teams. This work presented three aspects of knowledge relates to this issue:
(1) Ergonomic issues - most of collaborative virtual environments' design approaches are driven by technological constraints. The disadvantage of such approaches is that they produce virtual environments often far away from users' needs regarding collaborative activities.
(2) Awareness issues- there are known methods for evaluation of the awareness of users, which are more connected to the usability of the whole system than to the awareness subsystem. Quantitative measures are hard to execute, and the most widely applied methodology for evaluating user awareness of processes are questionnaires asking the users about their feelings. Using this methodology researchers were able to compare different awareness implementations, but not their effectiveness. Beside the evaluation of the effect of the applied awareness information via questionnaires, researchers could measure the productivity of the participants directly. From these measures they were able to collect some quantitative values about the effects of using different awareness types in their use case. Performance indicators were also important, but this methodology is not always applicable since it requires several independent target specific measures.
(3) Collaborative environment for real world applications - the proposed urban planning system identified the tools required to support the stakeholders' activities in the TF programme. These tools allow individual stakeholders to better communicate and share data with other team members as well as to support their own work. Urban information frameworks can allow the aggregation of various dispersed and distributed urban datasets from various organizational stakeholders to create a virtual prototype for an urban area. Modelling the TF as a case study assisted to define the objective of each organization involved in the program as well as the user activities required to build a sustainable community in order to combine data sets and enhance collaboration amongst the multi agents involved in the program.
Potential Impact:
The impacts of VISIONAIR are in multiple directions:
Fundamental research was directly impacted by the proposal of new resources “easy available” with technical support. Most technologies deployed by VISIONAIR are high end technologies, often prototype systems with a single existing configuration. Then investing and maintaining these solutions remains a research speciality by itself which is usually partially exploited with a limited number of research directions. The capacity for researchers to get benefits of these technologies without investment and maintenance issue was just an opening at the international level. Nowhere researchers who could need these techniques can access such a wide panel of resources without investing a big part of their activity in the engineering work. Here VISIONAIR radically changed this situation. The ramp-up in trans-national access projects demonstrate that it was not easy three years ago for an external researcher to understand the benefit of accessing these technologies for their own researches. But the first projects, first success stories create a kind of buzz. At the last general assembly in Rennes, a biologist already involved in visualization made evidence that he was practising these technologies as a very good handyman but that accessing Virtual Reality CAVE within VISIONAIR demonstrated the capacity to radically boost his research. He told that one day in such an environment could lead to one publication where with usual techniques several months where necessary for the same result.
Innovation in processes and practises is another direction where VISIONAIR demonstrated a clear impact. Technologies are existing and evolving very quickly. Partners of VISIONAIR participate to the development of technologies but a major contribution here was the test and validation of processes using the visualization and interaction technologies. We often mention the demonstration in ICT Vilnius where a dancer located in Grenoble danced on a music improvised by players located in Barcelona, Poznan and Vilnius. A 3D avatar of the dancer was produced in realtime and projected in a virtual scene surrounds by streams of the players. The virtual scene was projected in 3D to the audience in Vilnius. This is not science fiction nor deep fundamental research. Technologies exist but the strength of VISIONAIR was to be able to cluster the technological resources and to organize them for this single and amazing event. Such a demonstration is the proof that this process is really feasible. It brings the idea to a real innovation quite ready to go to the market place. The VISIONAIR individuals who were on the VISIONAIR booth at ICT in Vilnius made evidence of the impact of the demonstration for the witnesses. Many other TNA projects were submitted afterwards to repeat partially or fully or to overpass this demonstration. It is the same when we consider surgery tele-operation tested and awarded via the HYPERMED project. It is the same when we invite sportsmen to analyse gestures in a virtual reality CAVE. Projects to visually monitor Police or Firemen activity are anticipating the incoming concept of smart cities. VISIONAIR demonstrates the feasibility of the processes and open the way to the deployment of the technologies.
Opening facilities was also a matter to open the mind of VISIONAIR partners. It is a deep impact indeed to enter in a process where we must share the technology we developed for a very specific usage and thus to adapt our processes for other applications. First it ensured a more complete usage of expensive facilities which were sometimes poorly used. Secondly it enforced the creation of new interoperability process and it could be easily demonstrated that every partner passed a big step for organising projects. It thus also increase the maturity of partners respect to the technologies they maintain leading to a combinatory explosion of activities. It also highligth the need to create new integration platforms at other levels. VISIONAIR was a group of 24 partners and should integrate much more facilities. The interconnection of these facilities for a better research and civil usage is inherent of the development of the technology itself. That means that VISIONAIR his a “has been” model that should be widely extended in Europe. As it will be discuss just after we fully share the concept of community ring developed in the SPECIFI project (http://www.specifi.eu/) where a VISIOANAIR like facilty must play a specific role.
Industrial, economic and citizen impacts: as a scientific infrastructure VISIONAIR did not include non academic partners. But indeed a first link was clearly and naturally established with visualization and interaction device providers. Then links came with end-user and once again by the need to innovate and demontrate the possible deployment of specific use-cases. Almost 10 partners from VISIONAIR are directly in connection with industrial applications because their research field is the development of tools for production systems. Another set of partners are more directly connected to healthcare activities and the research conducted within VISIONAIR will be deployed in this sector. Not least a few partners were really involved within city applications: connections between artists, interdisciplinary collaboration, etc. Even if the duration of a project in VISIONAIR cannot directly lead to a patent, the infrastructure indirectly participate to innovation in all these sectors.
European attractiveness: VISIONAIR was conducted with a scientific board. We invited recognized scientists in this board, from the US, from Brazil, England and Germany. And obviously we had much more contacts with scientist and engineers outside Europe. We clearly share the expectation of an infrastructure model delivering research resources to all the scientific community. While network communication system are widely developed worldwide, while disciplinary infrastructure are created, we also need intermediary transversal support for the middle ware between hard communication and human expertise.
A Key Innovation Enabler Visualisation as a key enabler assists in dealing with typically complex challenges i.e. to first understand and subsequently manage factors, relationships and interactions. Complex challenges (e.g. environment; sustainable healthcare; economic prosperity) are likely to be relevant to different disciplines and cross national boundaries. Visualisation has a role to play from two perspectives:
(1) communication across participants in different locations, and
(2) to interpret and analyse data.
The findings reported in this final version of this deliverable, are based heavily on the internal and external recommendations gathered in the context of Task 1.3 “Internal and external interactions”. Internal interactions recognise specific existing technologies and the areas that they have been applied. At a high level the areas are classified based on industry sectors, whilst further analysis is required to be conducted in order understand the challenges. On the other hand, external interactions help to understand the challenges faced across industries and position correctly the current issues that need to be addressed.
Priorities for European Innovation: nowadays, most scientists usually practice in 2D visualization, which covers their main needs, therefore VISIONAIR needs to identify:
a) The real need for 3D applications
and
b) The communities that really need advanced visualization technologies.
Research priorities for advanced visualization should indicate the strategic areas in which the long-term vision of VISIONAIR should target and the directions it should follow within each of these areas.
Strategic areas: VISIONAIR has identified the following strategic areas for advanced visualization research:
Manufacturing: visualising data about the life cycle of engine/part/system both at the early design stage and operation. Illustrating, dynamically, forecasted data about time to failure and utilisation.
Design: accelerate product development, time-to-market and reduce prototype development costs by using virtual prototyping techniques.
Computer supported collaborative work: Open universities are becoming increasingly popular for people to continuously educate themselves. This means that individuals do not need to attend lectures and can utilise provided material or online courses as it suits them. Universities dispersed in different locations are collaborating and in order to deliver targeted outcomes. Visualisation has a role to influence the level of collaboration and the quality of the outcomes.
Cognition: Complex user studies that involve the cognitive analysis of humans can be undertaken in advanced visualization facilities.
Medicine: Governments around the world are challenged to reduce the cost of delivering healthcare services to patients. A trend has emerged that promotes preventative measures while surgeries are increasingly attracting interest to facilitate this. However, surgeries tend to be less equipped and overall may be less skilful to conduct various forms of examinations that are made in hospitals. A major characteristic of these examinations is to visualise the health of the patient and technologies that facilitate these in an affordable, user friendly (e.g. less skill requirements) manner. There are also opportunities to provide very detailed and accurate images of patients in an operation for a GP (general practitioner) in a distant location and to share data about health conditions in a secured manner.
Ergonomy: ergonomic studies for products and processes require complex tracking systems and advanced algorithms for measuring the user’s movements and behaviour. Ergonomy plays an important role in the design of many products and processes.
Biology, Chemistry and Nature observations: complex organisms and mechanisms of the nature cannot be replicated in a control environment without significant cost and time. Environment is a cross industry aspect, where there is a need to be able to compare and contrast the impact on environment from different kinds of production and service provision models.
Physics simulation: many applications require the simulation of the physics phenomena taking place during their observation. Visualizing the behaviour of materials under stress can provide inside in to their failure mechanisms etc.
Arts, multimedia and patrimony inheritance: museums, architectural design of urban spaces and other applications can be benefited from interactive and immersive simulation environments.
Future advanced visualization research and development follows certain directions based on the current trends and challenges. Therefore, it is important to be established which directions are relevant for each strategic area, so that it can be followed through a suitable business model.
By the end we are convinced that VISIONAIR fully participated to the visility European Reasearch Area and that an extension of VISIONAIR can continue with this role.
Main dissemination activities and the exploitation of results
Dissemination was mainly organized in the work-package 2 but not only. A lot of activities were directly driven through trans-national access contacts or peer to peer contacts on the initiative of specific partners. The strategy has been first to build a wide awareness about VISIONAIR but it was not enough. The matter was to convinced external users that it was not impossible to access the facility as soon as a reasonable project was proposed without out decreasing the quality of accepted projects. Then a second strategy step was to connect VISIONAIR to other facilities and projects. We enter in deep contacts and actions until with TNA projects with the SPECIFI initiative opening a huge area of activity in the direction of smart cities and collaboration rings. We enter in discussion whith PRACE about the interest of Visualization and Interaction within HPC. We enter in connection with observatories, and with scientific societies to create better links with communities.
A specific effort was conducted to participate within conferences and events concerning disciplines which were not represented in our TNA accepted projects improving the interdisciplinary activity of VISIONAIR. The JRAs also participated directly to the creation and the promotion of knowledge and obviously every publication was expected to cite VISIONAIR. The same for Trans-National access projects who led to a lot of common publications.
But indeed WP 2 dedicated to external outreach was the official organization of dissemination. The following sub-sections outline the targets for the specific tasks.
Task 2.1 - Web Portal and newsletter
This task aimed to sustain a web portal and newsletter. The web portal involved updating the website with up to date information about proposal submission, areas of interest, example TNAs and news items (e.g. EU deliverables, newsletters, and VISIONAIR events). The web portal has been fully functional, and supports with achieving some key targets such as building a dynamic relationship with external communities, attracting new proposals, disseminating outcomes of VISIONAIR (e.g. deliverables), promoting events, attracting institutes to the Partners Club and building a visualization community. In parallel, the intranet, which enables internal communication, has also been fully functional and has been successful at engaging over 90 colleagues. Task 2.1 has achieved the targets set out for the project. In the last period an area that was a significantly enhanced in the web portal was ‘Browse VISIONAIR Resources’.
Task 2.2 - Attending Trade exhibitions and international events:
The task 2.2 developed the strategy on joint external outreach at international events for VISIONAIR. As defined in the developed process models, the approval and information of Partners attendance towards international events was handled by the task 2.2 involved Partners. Advancing with new project results, the focus of attended international events was increasingly on the dissemination of JRA results and TNA projects. By sharing the experiences gathered during several TNA projects, external audiences have been introduced in a lively way to the opportunities, facilities and know-how offered by VISIONAIR. The presentation of JRA results at international events increased the visibility of VISIONAIR at international scientific level. Hence the presentation focus was not only the sheer research results, but also the activities regarding networking between partners and building up a shared infrastructure.
The successful cooperation between the VISIONAIR partners and external researchers about TNAS lead to attendances at events which directly showcased the capabilities that the TNA model offered. As direct result out of TNA projects joint papers and attendances to international events were realised, whilst incorporating VISIONAIR partners and the external TNA guests.
In addition to the regularly attended international events of the Partners (e.g. conferences to introduce TNA and JRA results) task 2.2 worked on the attendance towards key events on which VISIONAIR contributes to larger extend. These key events are determined by demonstration activities (such as demos, booths…) to highlight the capabilities offered by the consortium. By addressing such events the visibility of the project could be enlarged.
In the last year 48 international events have been attended by VISIONAIR Partners, to publish results (TNA results, JRA results and additional results directly related to the project) but also to do promotion among the capabilities of the project. Thereby the active participation (both, demo, sponsorship…) was the central measure to promote TNAs and raise attractiveness of VISIONAIR. By showcasing the interconnectivity of VISIONAIR partners, but also the involvement of external TNA participant, the project could be represented in an extraordinary way. The list of events were VISIONAIR attended is obviously fully reported in the periodic reports and is not reproduced here.
Task 2.3 Invitation & setting up for external proposal
This task is involved making sure the website was up to date with the call for proposals, the template form to submit proposal, regular proposal review meetings, request and collate reviewer feedback, and managing the completed TNAS.
Led by Cranfield, the review committee included representatives from Grenoble INP, HLR and Technion, Israel Institute of Technology. The “Proposal management/selection” area in the VISIONAIR site, managed by Grenoble, facilitated the operations of the committee. The targets for this task have been achieved.
The process of collecting the feedback from the host and the reviewers for a visit involved standard templates which were designed and used in this process to collate the information.
Task 2.4 Open forum development
This first open forum was held at Grenoble INP during the kick off in March 2011. The second open forum took place in Poznan in 2014. It was an extra day associated to the kickoff meeting or annual general assembly, fully open to any interested person when we mainly presented goals, and projects results. It has been two powerfull dissemination days.
Task 2.5 Organise an Associate Club
The Associate Club members are listed on the website and of four types: industrial, academic, visualisation vendor, scientific contributor. There were a number of ways to engage with the Associate Club members. For example, we shared different videos registered during the VISIONAIR seminars in order to enhance their knowledge about VISIONAIR activities and resources.
Members of the Club were also invited to participate to special event during conferences with specific VISIONAIR sessions.
They also have been invited to participate during the 1st General Assembly in Twente, the 2nd in Stuttgart, the 3rd in Budapest and the 4th in Rennes.
The 2nd to the last years has been devoted to the enlargement of the Club of Partners with the integration of EMIRAcle as a member of CVNET, the “Color and Vision Network”. On the flow of messages transmit in CVNet, EMIRAcle selected the interesting message for the VISIONAIR members, transmitted the messages to them and to the Club of Partners, and invited the expeditor of the message to reach the VISIONAIR Club of Partners.
As a network of Higher Education institution, EMIRAcle has the deal to promote VISIONAIR for their members who are not already partner in the VISIONAIR Infrastructure (13 EMIRAcle members on 25), and this is mainly outside Europe. The special links between EMIRAcle and the Asian countries permits the dissemination of the VISIONAIR infrastructure in Thailand and in India.
A first keynote has been given by Pr. Serge Tichkiewitch during the Conference for the 56th anniversary of the King Mongkut University of Technology North
Bangkok, the 19th of February 2014 (Figure 7) with title: “The European VISIONAIR Infrastructure”. Among the participants was the member of the Design, Manufacturing and Innovation (DMI) network, a network of 15 Thai Universities on the
governance of the National Science Technology and Innovation Policy Office (STI), Bangkok, Thailand, STI being one of the EMIRAcle members. The result of this conference was a proposition for a Program in Higher Education and Research SIAM 2015 between the Chiang Mai University (Thailand) and VISIONAIR Grenoble (France) on “Virtual Cultural Heritage - Digital Artifact Representation
in Collaborative Environment”, accepted in January 2015.
Using the capacity of the SCITE application to disseminate the research papers written by the members of the VISIONAIR Infrastructure and of its Partner’s Club, we add a special Web page about “Bibliography on Visualization” on the VISIONAIR web site, with the capacity to get the papers sort by Titles, Authors or Keywords. 77 papers concerning 186 keywords and written by 255 authors are available on the actual site.
List of Websites:
www.infra-visionair.eu
VISIONAIR is organized through a technological oriented vision. Our 29 facilities were associated to four technological domains: 1/ Augmented collaborative environments, including holography, were supported to enable local or remote collaboration of actors. 2/ Ultra-High definition and Networking facilities provided UHD displays (4K, 8K even in 3D), including streaming the corresponding content on networks. 3/Virtual Reality facilities are related to CAVEs and other immersive environments where the user is immersed in virtual scenes. 4/ Scientific visualization facilities were dedicated to the navigation and understanding of results issued from High Performance Computation or from nature and physical observation.
VISIONAIR hosted 122 external projects from the start of 2012 to the start of 2015, covering a wide spectrum of disciplines as:
• Improvement of Engineering processes: interacting with fluid dynamics simulations, weather forcast, new style design, extended vision and perception of technical 2D drawings, manufacturing process analysis or simulation, augmented tools for manufacturing, manufacturing plant layout arrangement, training on assembly and disassembly of products.
• Remote collaboration: remote physical feedback by handling haptic device from a long distance, co-working sharing 4K panels, co-organisation of nurse schedule, surgery tele-operation; and also remote collaborative concerts involving musician and dansors located in various countries and participating to a same concert within a virtual environment.
• New tools and methods for medical applications. Ergonomy analysis to support sport analysis activities. Training gun fighters, analysing rugby-man or basket-ball gestures or enabling new protocols of remote interconnections between athlete and coaches. It was efficiently demonstrated the interest of 3D ultra-high definition quality with very low bandwidth and very low latency over networks to support this kind of remote collaboration.
• Museum applications were realized. The Last Supper from Leonardo Da Vinci was reconstructed in 3D to extend its perception, reconstruction of fragile historical sites for patrimony conservation and archeological studies.
• We also organized a virtual visit of Mars with data incoming from observatories, exploring seas and animals scanned from tomography and so much more including studies about virtual environment perceptions, pyschology, etc.
VISIONAIR is very proud of all this variety of activities which consolidated and extended a community practising the newest visualization and interaction technologies.
Project Context and Objectives:
The VISIONAIR infrastructure is a network of high level visualization and interaction facilities available for the European Research community. These facilities require expensive investment, maintenance and preparation work to enable innovative usage. VISIONAIR expects to share this effort made by teams in various location spread in Europe to offer more access to scientists and users who cannot make the corresponding investments. The partners of VISIONAIR were issued from several communities represented by FP6 network of excellence (VRL-KCIP, Intuition, aim@shape) but also from partners in connection with other infrastructures (HPC-Europa, Network providers, etc).
When starting this infrastructure in 2011, VISIONAIR was clustering 24 partners involved in 29 facilities. These facilities were mainly disconnected. Visualization and Interaction technological domains remain poorly standardized. For 3D visualization, most partners are either using some commercial solutions or developing their own software solution to support use-cases to be achieved. University of Stuttgart (HLRS) developed the COVISE engine; Sztaki developped VIRCA, INRIA Rennes developped OpenMask. Grenoble-INP developped CVE and no commercial solutions (Unity, 3DVia-Virtools) or open-source project (Blender, Ogre) can be considered as a standard. Visualization and Interaction is also supported by Ultra-High definition and streaming technologies and if the market defines standards (Full HD, 4K, incoming 8K, JPEG2000 protocol), here again every use case cannot be achieved out of the box. Complex and adapted connections must be engineered. Last but not least no real repository was existing to share models and tools about this wide domain. Because the 29 initial facilities were disconnected, the partners were mainly creating local demonstrations about training, expert assistance, telepresence, collaboration, streaming, cognition issues within immersion but the creation of benchmark use case and comparison data base was just a will. A few interactions between facilities were organised through various projects but the result were hardly consolidated.
VISIONAIR has been first organized through a technological oriented vision. Our 29 facilities were associated to four technological domains:
• Augmented collaborative environments, where every augmented reality technology, including holography was supported to enable local or remote collaboration of actors.
• Ultra-High definition and Networking facilities provided UHD displays (4K, 8K even in 3D) which was the first facilities in Europe at this level of resolution, but also streaming the corresponding content on networks.
• Virtual Reality facilities are related to CAVEs and other immersive environments where the user can immersed in virtual scenes.
• Scientific visualization facilities were dedicated to the navigation and understanding of results issued from High Performance Computation or from nature and physical observation.
The knowledge about these facilities was unclear. VISIONAIR has then developed an E-Resource map which is the first inventory of the 29 facilities. It is a first important result of VISIONAIR. Indeed such an action has two effects. Obviously, it creates an awareness capacity because the devices can be exposed to a wide public but it expected also to create an ontology of resources which drives a common vision towards a standard classification of devices. This E-Resource map is operational and could be open to a wider access.
The E-Resource map also promotes the facilities towards external interested users. As an infrastructure a main goal of VISIONAIR is to open access to the best researchers from any domain of competencies. The TransNational Access model (TNA) was a major activity. A permanent call for project was organised from January 2012 to end of January 2015. A committee review was in charge to assess the submitted project with a peer review process including a technical review about feasibility and two scientific reviews: one from an internal member of the consortium and one external review from the discipline of the application. During this period VISIONAIR registered in its database 208 projects and 122 projects have been successfully completed by the end of the project. That means 122 new use-cases were experimented with the support of VISIONAIR. All the completed projects were reported in the Visionair Application Management (VAM) database with a minimal format including the question of research, a presentation of the achieved process and the main conclusions of the project.
With TNA projects, VISIONAIR demonstrated its capacity to support any discipline and to become a major location for multidisciplinary research activity. VISIONAIR covered a wide spectrum of disciplines with:
• Engineering applications: immersion in fluid dynamics simulation for engineering improvement of product design, immersion in weather forecast, style design activities for artists and designers, extended vision and perception of technical 2D drawings, manufacturing process analysis or simulation, augmented tools for manufacturing, remote collaboration around the arrangement of a manufacturing plant layout, training assembly disassembly of products. But engineering needs often collaborative practises: handling haptic device from a long distance, co-working in front of 4K shared panels, co-organisation of nurse schedule, tele-operation for medical solutions.
• New tools and methods for medical applications. In this field ergonomics is a traditional activity including visualization and interaction devices. But VISIONAIR was also expected to support sport analysis activities. Training gun fighters, analysing rugbyman or basketball gestures or enabling new protocols of remote interconnections between athletes and coaches. It was efficiently demonstrated the interest of 3D ultra-high definition quality with very low bandwidth and very low latency over networks to support this kind of remote collaboration.
• Artists were also motivated to use VISIONAIR facilities. In ICT Vilnius in December 2013, VISIONAIR supporting the SPECIFI project demonstrated its capacity to create a remote collaborative concert involving a pianist in Barcelona, players in Poznan, and one last musician in Vilnius. They played together in real-time with a dancer improvising in Grenoble. The dancer was in a CAVE, and a 3D avatar was created in real-time allowing the synthesis of a 3D scene for the audience in Vilnius. Furthermore some applications were realized to improve story telling around the last super from Leonardo Da Vinci in collaboration with the Last Super Interactive project thus leading to museum innovative applications.
• About museums, VISIONAIR was also involved about patrimony conservation creating virtual mock-up of fragile historical sites or organizing archaeological studies in virtual environments.
• Architecture applications, support to mathematicians who expected to visually check some mathematics assumptions, but also virtual visit of Mars with data incoming from observatories, exploring seas and animals scanned from tomography and so much else including studies about virtual environment perceptions, psychology, etc.
VISIONAIR is very proud of all this variety of activities which consolidated and extended the community. The European Celtic excellence price discerned to the HyperMed TNA project came to consolidate the proudness.
An internal seminar where the 24 partners were invited on a regular base (either every two weeks or monthly) was also organized. This seminar was the opportunity to share common knowledge and to improve the strength of the emerging VISIONAIR community. In the first year (2011), when no TNA project was applied the seminar was the opportunity to discover the VISIONAIR facilities. Then starting with the first TNA projects, the seminar turned into technical tutorials to demonstrate specific use of the facilities than to present the TNA results. Every seminar presentation was recorded and is available from a broadcast server: it delivers 65 tutorials. This is an important networking activity which created a capitalized output. Reports and recorded tutorials are a reach database of benchmarks and use-cases which is shared widely from our website.
VISIONAIR makes sense as soon as its TNA projects are organised and capitalized for potential future re-use. The ontology issued from the aim@shape project, initially dedicated to share shape models, was extended to visualization and interaction domain and was also connected to the Eresource map. Then tools referred in the new VVS ontology (VISIONAIR Visualization System) are directly associated with the facilities which deliver them. New protocols were also implemented and tested to enable remote usage of the corresponding algorithms. To avoid an increasing number of information sources every information from the VAM database (TNA projects), from VVS system, from the Eresource Map were merged in the VISIONAIR Browser which acts as a single entry point to all the VISIONAIR results
(http://www.infra-visionair.eu/what-we-offer/visionair-resources.html). VISIONAIR browser provides first access to content through a discipline filter. From a discipline the corresponding facilities, TNA projects and adapted tutorials are proposed. But the browser offers direct access to the TNA projects, tutorials and resources (facilities). The list of available service are also directly listed there. The summary of joint research actions (internal research combining several partners) and information about our club of partner are available there.
The Browser demonstrates the capacity of VISIONAIR to create synergies and enable new type of interconnections. The facilities were initially mainly disconnected but the VISIONAIR project was a unique opportunity to interconnect them. Here things are not solved: interoperability is still an issue indeed but the partners learned and experimented a lot during the past period. Many TNAs expected the support of several facilities as the Art Concert experience mentioned previously. TNAs enforced the movement which was also investigated within the Joint Research Activity. How to design a manufactured product in Barcelona and to get a 3D feedback on an holographic display in Stockholm. How to connect an immersion CAVE in Rennes and a second one in London, or between Image Institute in Chalon sur Soane and another CAVE in Madrid, how to process information from various sources? Many demonstrators were developed and created new reference knowledge for these activities. Indeed by the end, VISIONAIR developped a real networking knowledge. Joint research actions were decomposed into 3 work-packages. One about hardware, one about tools and methods, and one about assessment of the usage and impact of technologies. Twenty internal research actions have been completed, extending the overall knowledge of the VISIONAIR group. The link between engineering CAD models and visualization and interaction was investigating creating new services enabling the automatic recognition of kinematics joints in CAD model to prepare virtual worlds. The orchestration of remote algorithms was also successfully tested, new 3D and 2D metaphors were compared and classified respect to usage, new algorithms to improve real-time remote presence, new methods for large navigation in restricted immersive environments, new tangible interactors were investigated and tested. Usage of newest holographic table with remote connection was demonstrated and completed the activities about interoperability among distributed platforms. Modeling (and support to) user requirement and user engagement, assessment of collaboration work, use of model driven applications to develop collaborative environment were also investigated in JRAs. All these activities were directly complementary to the needs observed within TNA projects and JRAs experimented new services that could be developed or become operational for new applications.
The community is still strongly alive. This knowledge must be maintained, developped and opened. The partners of VISIONAIR contacted in many directions other infrastructures and communities to go ahead. A Special Interest Group of the EURO-VR association was created in December 2015 to support this activity while opening it to a wider participation. The intention remains to build technical interconnections between distant facilities to open access to new applications. Visualization and interaction must become a core service as internet networks are considered now. Over the network end-user need tangible, intuitive, and high quality visualization and interaction models. There is thus an intermediary layer of tools, methods that must be maintained and developed. VISIONAIR is clearly a first instantiation of such a layer. VISIONAIR has been in direct contact with HPC-EUROPA, PRACE, EMIRACLE, CIRP, Design Society, SPECIFI, VFF (Virtual Factory Framework), EPOS (european planet observing system), CINEGRID, etc.
No doubt that the story is not ended!
Project Results:
Overall presentation of S & T results
As an infrastructure the main goal of VISIONAIR was to support TNA projects. 122 projects were completed supporting external researchers who expected to use the VISIONAIR facilities for their projects. It is our major result with a lot of steps passed in many fields of research. To enable these TNA activities, coordination actions expected to solve some technical issues supporting the consortium integration. These developments were mainly conducted in the work-package WP3. In addition the joint research activities are sources of science and technological results. They were organised in 20 actions to pass over technical and scientific challenges which will improve the VISIONAIR support to TNAs.
This part follows the work-package organisation to report the main scientific and technological results. For many actions the inter-relations between work-packages was important: some TNA use-cases were directly re-used in Joint Research Activities which were enabled by coordination actions. This may lead to a few repetitions in the following sections. First we present the coordination activities results. Then the Trans-National access results are reported to finalize with joint research activity reports.
Coordination activities
Scientific and technical issues within coordination activities were associated to work-package 3. This work-package was organized into 4 tasks dedicated to improve technical and coordination integration within the consortium.
o T3.1.1: Video-conferencing/communication:
The task 3.1.1 was in charge to ensure communication between partners. Many solutions exist and have been clearly improved during the VISIONAIR project duration. It was not so easy four years ago to animate a project as VISIONAIR with an important usage of video-conference systems. The various solutions were benchmarked within this task and the EVO system was selected. It was not the better quality system but it was free for scientific communities and had the big advantage to be ported on most platforms. Indeed it was encoded within a java platform independent solutions.
At the end of year one the EVO providers changed their business model to turn it into a commercial solution, named SeeVogh. A new benchmark was operated inside VISIONAIR. Because RENATER was supporting a SeeVogh Community, and because Grenoble INP is a referenced user of RENATER, it was possible to provide access for VISIONAIR partners to the RENATER community within SeeVogh servers. During period 2 and 3, Grenoble INP created local accounts for every partner to provide access to the commercial solution SeeVogh which is deployed by RENATER. The usage of the SeeVogh network works in a satisfying way; no additional efforts are (or need to be) employed. Although SeeVogh has a more commercial approach as compared to the EVO solution, the current working method is certainly feasible for all Visionair project partners. Meetings, seminars and presentations run smoothly, almost without being hampered by technical issues.
If SeeVogh was satisfactory we still experienced technical issues with some partners. Mainly due to a lack of preparation of the work stations. It is a state-of-the-art result. The technology had an impressive enhancement in the last four years but the connections must be checked, specific loudspeaker or microphone must be used to avoid echo and disturbance.
It must be noted that most coordination activities where coordinated through video-conference, saving a big budget and time for travels. It is thus a very positive experience where the VISIONAIR consortium demonstrated its capacity to use intensively these solutions which is technical result. We just learned that RENATER stopped its participation to SeeVogh but our benchmark analysis let us with plenty alternative solutions with a growing maturity.
o T3.1.2: Broadcasting techniques:
Broadcasting techniques were not new. The main technical issue was to define a process to use existing solutions for the promotion of our internal results. No additional effort need to be employed unless to follow the process as soon as we decide to publish a video.
We intensively practised broadcasting. Some TNA projects were reported via videos and the internal seminars conducted by video-conferences were recorded and reworked to provide a publishable video. These video tutorials are created by UPatras. Grenoble INP was in charge to upload the videos on the University of Grenoble broadcasting server. Then the link is shared and deployed on the website. In parallel, Videos were posted within YOUTUBE to ensure a wider promotion of VISIONAIR actions.
Here again it is a good success of the VISIONAIR activity. More than 65 videos from 5 minutes to one hour were produced and are available from the VISIONAIR website and usually in the VISIONAIR group in YOUTUBE.
o WP3 T3.2 Synthetic environments:
A key point about visualization and interaction technologies is that they are not fully plug and play. Every use case requires new development and the potential to imagine new use-case is very high. An infrastructure supporting access to as set of heterogeneous facilities needs new adaption and developments for every new use-case. It must be a collaborative process involving the final user and the staff in charge of connecting the technical resources. The University of Twente developped the concept of synthetic environment. It is indeed a road-map system taking in charge the main steps of the collaboration from the initial demand to the delivery of the specified system.
A web tool was thus developped and open for usage by TNA-projects. The employment of the roadmap has been demonstrated during the General Assemblies and during a VisionAir seminar.
A significant number of partners indicated advantages of the roadmap but most partners still apply this process in a more informal mode. Some best-practices knowledge have been integrated in the synthetic environment roadmap; Given the number of page hits, many partners do take inspiration from the roadmap, but do record their prepararory processes in a different manner. The road-map contributes to the best-practices being availabe and takes away the fear that the first few best-practices attract unbalanced amounts of attention. The road-map remains a service available from the VISIONAIR portail for any other project.
o T3.3 : Interoperability issues
Interoperability challenges are still very important. The creation of an infrastructure like VISIONAIR was obviously facing the corresponding traditional issues. First because the facilities are spread in Europe and were not initially connected. VISIONAIR project was a unique opportunity to interconnect them. Interoperability is still an issue indeed but the partners learned and experimented a lot with VISIONAIR. Many TNAs expected the support of several facilities. In ICT Vilnius in December 2013, VISIONAIR supporting the SPECIFI TNA project, demonstrated its capacity to create a remote collaborative concert involving a pianist in Barcelona, players in Poznan, and one last musician in Vilnius. They played together in real-time with a dansor improvising in Grenoble. The dansor was in a CAVE, and a 3D avatar was created in realtime allowing the synthesis of a 3D scene for the audience in Vilnius. Indeed TNAs highlighted the demand and in many case Joint Research actions were expected to solve this issue. How to design a manufactured product in Barcelona and to get a 3D feedback on an holographic display in Stockholm. How to connect an immersion CAVE in Rennes and a second one in London, or between Image Institute in Chalon sur Soane and another CAVE in Madrid, how to process information from various sources?
Benchmarks were experimented. Arts et Métiers Paris Tech provided a full CATIA model of a complete real plant. The challenge was then to find the best way to import this very huge model on various Visualization sites. Several guidelines and tools were developed by HLRS and Grenoble-Inp about this case and overpassed the challenge.
Another usual interoperability barrier concerns the internet connections of two sites because of security issues. A VISIONAIR virtual private network was implemented by I2CAT to offer network connections free of firewall issues and was provided as a technical service for any need.
Many demonstrators were developed and created new reference knowledge for these activities. Indeed by the end, VISIONAIR developped a real networking knowledge. The first results highlighted the existence of many solutions to ensure interoperability between facilities. Nevertheless, use cases based on real scenario have shown some issues in using those solutions. Guidelines were produced for several specific domain where transforming domain data to visualization environment is expected.
o T3.4 : Virtual service infrastructure adaption for visionair needs
This task mainly concerned the evolution of the VVS service that is operated within WP8.
The VVS system is an evolution of Digital Shape Workbench, an ontology-based repository of shapes, tools and ontologies developed in the aim@shape FP7 network of excellence, coordinated by CNR. The DWS server was moved the CNR facilities and its maintenance was taken in charge by CNR. Within VISIONAIR, the shape and tool ontologies were extended to cope with Visualization and Interaction issues, the search and browsing capabilities were extended and improved turning the system into a Virtual Visualization System.
In WP1, a E-Resource map was developped by University of Patras. The role of this service was to deliver a catalog of all the VISIONAIR resources. This second ontology system is a first opportunity to declare, and promote the technical tools owned by the partners and it becomes a reference data-base in Europe.
Both CNR and Upatras collaborated to interconnect the two ontology. Especially the software resources owned by the facilities are shared by the two systems and are synchronized accordingly.
Technically VVS is first a huge repository of 3D shapes. It provides capabilities to enable content-based retrieval of 3D shapes. Hence, without using meta-data description, VVS may find similar shapes by direct comparison of models. The data repository was extended with new models including the creation of a specific manufacturing oriented view of the shape repository, thanks to the ontology defined in the JRA in WP9 which also guarantees connections to the VFF IP project results working to the definition of a Virtual Factory Framework.
VVS is also a catalog of algorithms and tools. Some joint research actions extended the VVS with a new repository of shape processing workflows to both provide informative material and to include the possibility to directly executing shape processing operations as services. .
In the last year the number of access to the VVS was doubled showing the interest for the platform and the results of a good dissemination of the VVS and of VISIONAIR. We can also note that the % of increment of visits of the shape repository (132%) is much bigger than the tool and ontology repositories (respectively 90% and 84%) showing the large interest in reusing shapes. Moreover most of the accesses are from Europe (51,43% for the shape repository, 63,66% to the tool repository and 56,51% to the ontology repository according to ClusterMaps© and 49% according to Google Analytics) followed by Asia and then North America.
These results demonstrate the high level quality of VISIONAIR services. Some over services were built and delivered from the website. To maintain a single portail to these services, Grenoble INP developed and maintains the VISIONAIR Browser which provides an integrated access to all the services and create relations between the resources in the E-Resource Map, the Trans-National Access Project, the VVS system, the broadcasted tutorials and promotion videos and the results of joint research activities.
Trans-National access results
The main result about Trans-National Access is that VISIONAIR supported 122 projects during the 3 years were TNA support was active. The ramp-up has been quite important. VISIONAIR was in a way a new infrastructure even if facilities were already existing before VISIONAIR. Before VISIONAIR most partners did not practise a kind of service activity and they were more involved in research. The first year was - as anticipated in the initial description of work - a period of adaptation of the corresponding processes. It was a hard but very positive process for every partner. The creation of the committee review, the definition of rules and criteria to apply and accept projects, the process to capitalize what is done and considered as completed, the capacity to better formalize a facility activity is a real added value for most partners.
Then following the first call for project the first proposals arrived. 6 projects were completed during the last 6 month of period 1. We faced some mus-understanding about the regulation to fund projects coming from outside Europe and it was mandatory to revise our promotion to remove access from external applications. At the end of period 2, 18 months later, 50 new projects were completed and during the last 12 months, 67 projects were completed. The acceleration was obvious. It can be noted that we received a set of applications which were either continuation of a past project or submission of a new project from the same applicant. These cases demonstrate that the corresponding applicants were convinced by the VISIONAIR support.
At the end of the project we were in a stage when it became mandatory to reject some incoming projects because of the deadline, and to cancel some accepted projects because we did not have enough time any more to schedule the projects. We still receive demands, three months later, and we have clear indications that some projects initiated in the format of TNAs go ahead outside VISIONAIR model.
Many projects were very long to start. During the VISIONAIR program submission we sub-estimated the effort expected to prepare projects. Several TNA projects expected 2 years of remote collaboration before the formal invitation of guests. This long process was due to un-maturity of technologies and the necessity to adapt the facilities to the demands. This fact mainly explains the big activity in the last year when project were now read. The authorization from the last amendment (covering the last period) to take into account some preparation unit of access was a good option to measure a real effort associated to every trans-national access project.
The European Celtic excellence price discerned to the HyperMed TNA project demonstrates that VISIONAIR has been recognized through its support action, Obviously the VISIONAIR consortium is proud of this award, but also proud that VISIONAIR supported a very wide variety of disciplines and challenges. We covered :
Engineering applications: immersion in fluid dynamics simulation for engineering improvement of product design, immersion in weather forecast, style design activities for artists and designers, extended vision and perception of technical 2D drawings, manufacturing process analysis or simulation, augmented tools for manufacturing, remote collaboration around the arrangement of a manufacturing plant layout, training assembly disassembly of products. But engineering needs often collaborative practises: handling haptic device from a long distance, co-working in front of 4K shared panels, co-organisation of nurse schedule, tele-operation for medical solutions.
New tools and methods for medical applications. In this field ergonomics is a traditional activity including visualization and interaction devices. But VISIONAIR was also expected to support sport analysis activities. Training gun fighters, analysing rugbyman or basketball gestures or enabling new protocols of remote interconnections between athletes and coaches. It was efficiently demonstrated the interest of 3D ultra-high definition quality with very low bandwidth and very low latency over networks to support this kind of remote collaboration.
Artists were also motivated to use VISIONAIR facilities. In ICT Vilnius in December 2013, VISIONAIR supporting the SPECIFI project demonstrated its capacity to create a remote collaborative concert involving a pianist in Barcelona, players in Poznan, and one musician in Vilnius. They played together in real-time with a dancer performing in Grenoble. The dancer was in a CAVE, and a 3D avatar was created in real-time allowing the synthesis of a 3D scene for the audience in Vilnius. And that's not all, some applications were realized to improve story telling around the last super from Leonardo Da Vinci in collaboration with the Last Super Interactive project thus leading to museum innovative applications.
About museums, VISIONAIR was also involved about patrimony conservation creating virtual mock-up of fragile historical sites or organizing archaeological studies in virtual environments.
Architecture applications, support to mathematicians who expected to visually check some mathematics assumptions, but also virtual visit of Mars with data incoming from observatories, exploring seas and animals scanned from tomography and so much else including studies about virtual environment perceptions, psychology, etc.
This list could be extended with the fine description of the 122 completed projects, but the main important result is the capitalization of all these new use-cases. Visualisation and Interaction is definitely a trans-disciplinary resource which remains less mature than network facilities and thus require more support to enable a specific use case. The capitalization of the use cases creates a knowledge data-base that will remain published on the VISIONAIR portal for any reuse, comparisons and enhancement.
The trans-national access activities were split into four work-packages for management convenience. This four work-packages are presented hereafter. The boundaries between work-packages was not closed and indeed it was often difficult to associate the projects to the work-packages. This surely mean that this convenient decomposition which was originally due to sub-communities between partners does not make much sense or that VISIONAIR succeeded in a good integration of its internal communities.
• Scientific Visualization
Scientific Visualisation encompasses a wide area of fields including – but not limited to – engineering, medical visualisation, chemistry, astronomy, physics or biology. Essentially the mission of the work-package 4 about Scientific Visualization is to support scientists in visual analysis of their data typically produced by simulations or computations on abstract mathematical models. By providing visualization methods and technologies as well as improved analysis and modeling possibilities by the partners involved in WP4, transnational researchers can make use of interactive visual representations of their data. VISIONAIR scientists completed 29 projects within WP4.
Target groups for the transnational access may include:
Scientists that want to try out their visualisation methods or systems in truly immersive environments.
Scientists that want to connect their simulation code to high-end visualisation platforms to do on-line simulations with real time assessment of the simulation results.
Researchers who need high definition, high resolution output devices they usually do not have access to.
Scientists that develop novel post-processing methods or create parallelised versions that require a large computing effort.
Scientists that want to visualise massive data sets that are usually too large for common visualisation facilities.
Guests of the transnational access programme will not only have an extensive visualisation platform at their disposal, but also further resources that are required to undertake their visualisation tasks like supercomputing resources, high performance networking environments, experimental facilities and others. Although these resources exist and are available to the programme participants to a limited degree, proposals are required to be visualisation centred and should only in rare cases draw heavily upon these.
The projects comprise visualizations of huge data sets from simulations with the need of a high degree of interaction capabilities, researchers increased insight into their data by using innovative visual analysis and modeling techniques and improve research results and outcome in various research fields. Not only using scientific visualization, but some projects also focused improvement of visualization techniques for application in education or perception and awareness of 3D content focusing scientific visualization in general. Some projects also focused improvement of visualization techniques for medical or forensic applications for instance, which will further be used in training, education and research and further improve specific applications and scientific visualization in general.
It is not the purpose to cite all the projects here but we can cite the construction of a virtual representation of Mars built with the data coming from observatory enabling by cave immersion a transportation of scientist on Mars. This project was enabled at the Thinklab in Usalford. A more conventional project which comes with very nice image production concerned the evaluation of the turbulent incrompressible flow and the particle tracking in the Axialzyklon wih a high-resolution model. The scientist were immersed in a flow of visible particles to better understand the results of simulations. The ligth sheet experience is another project were the support of immersion demonstrated a very high interest for scientist. Biologists were able to clearly navigate inside insect and to better analyse the insect structure. More rapidly and easy that never before: during the last general assembly, the guest Dr. Emmanuel Reynaud claimed that accessing an infrastructure like VISIONAIR was a strong opportunity for biologist to produce new publishable knowledge.
• Ultra High Definition Imaging distributed on optic network
The infrastructure offered by the partners in UHD-NET (work package 5) provides facilities for visualization of scientific experiments. Among the others, the following research projects have been identified as potential users of the infrastructure:
Telemedicine, e.g. experiments with streaming from hospitals (operating rooms) for remote diagnosis and support,
Radio-astronomy, e.g. experiments with streaming high resolution pictures to remote participants for analysis,
Chemistry/Biology, e.g. live streaming from experiments to remote audience,
Cultural sector and sport, e.g. experiments with real time transport and visualization of important cultural and sport events, enabling research and innovation in cultural interchange experimentation by combining culture and high-technology,
Physics, e.g. real-time and offline scientific visualization using high resolution, e.g. remote rendering of scientific processes and simulations.
Other potential services and appliances of the infrastructure could be:
Telepresence and videoconferencing in very high resolution, excellent quality and low latency (uncompressed HD streaming),
Education and e-learning, e.g. high resolution transmission of lectures and presentations to the remote audience,
Entertainment, e.g. digital cinema with on-demand video transmission from remote location, gaming and immersion in virtual reality (3D).
In recent years there was rapid development of UHD devices, so the technology became more and more popular. The technology became also much cheaper, however professional devices are still expensive and unavailable for smaller projects or scientists, especially considering streaming UHD/4K technology. Many European projects and researchers started their interest of new technology related to UHD, 4K, 8K and stereoscopy, so that work-package partners (WP5) had much bigger interest from the potential users.
The principle of User Access was udapted so that some facilities could be be moved towards the applicant when necessary.These conditions enabled new TNA projects to be conducted and completed with a new model where the host becomes the guest and carries the necessary facility to the guest.
Most of the 19 Trans-National Access projects completed in WP5 was ultra-high definition streaming experiments for various scientific appliances or scenarios, just to mention: medicine, robotics, e-learning, sport, arts and culture. WP5 is composed of 4 installations, which are located in: PSNC (Poznań, Poland), University of Bristol (Bristol, UK), I2Cat (Barcelona, Spain) and University of Twente (Enschede, Netherlands). It must be noted that initially a team settled in UESSEX was involved in VISIONAIR but a part of this team moved to Ubristol. Then Ubristol replaced Uessex in the consortium. All installations are equipped with Ultra High Definition or 3D facilities and are interconnected with high speed optical networks (GEANT, GLIF, HPDMNET or other). Installations may be used separately or combined for complex scientific experiments. All the facilities are capable of supporting very high quality distributed visualization applications. A new and unique capability is offered by the combination of advanced visualization technologies and network services. The network services enable multiple high-resolution digital-media streams to be transported among global sites, including University of Bristol, PSNC, I2CAT and University of Twente, using dynamically provisioned optical light paths across multiple domains, which can be used on a scheduled or on-demand basis. The infrastructure offered by the partners in this WP provides facilities for visualization of scientific experiments.
WP5 partners showed that they are ready for common collaboration. Techniques, methods and integration of technologies, they worked on for last 3 years, allowed for making more complex experiments at the world level. With support of WP5 partners some of them were the first demonstrations of cutting-edge technologies in the world – for example Human-robot interaction over 40G network with UHD 3D High-Frame Rate at distance 2000km with minimum latency (TNA 114).
Some projects like project n° 136 (Research on new enablers for artistic virtual environments and performances) linked both WP5 and WP6 (using UHD-NET and VR installations). It demonstrated the capacity of VISIONAIR to integrate facilities together and provided really amazing collaboration (this project has already be mentionned sooner and concerns the organisation of a concert with players and dancers spread in four different locations in Europe). The Work-package allowed also the Hypermed project which investigated the use of 3D Ultra-High definition and high quality streaming for tele-medecine application. As already mentionned, this project was awarded by the Celtic excellence program.
This work-package developed a particular model of trans-national access within VISIONAIR with very impressive results. Tele-presence was also used to support sportive gesture analysis. The sportsman or group of sportsmen is working alone but observed in 3D by a remote coach. The coach can record the session and enter in close collaboration with the sportsmen even from a very far location. This kind of environment is quite new and expects a strong technical support. In all the use-cases, the final users, sportsmen, coaches, artists gave evidence about the new opportunities offered by the corresponding environments. It was also demonstrated the importance of the scenario organization by the involvement of end-users: WP5 created a space of convergence between endusers and technology developpers.
• Virtual Reality
The Virtual Reality facilities propose immersive environments. This is a huge work-package clustering a important number of facilities. Mainly virtual reality CAVE are concerned here, but also usage of interaction systems as haptic devices. The immersion capacity is also enable by Head Mounted devices and involve high level tracking solutions. The specific case of the GRIMAGE cave in INRIA grenoble creates a scanning solutions and has the capacity to produce in real time a 3D avatar of the observed scene. Many CAVEs were proposed by VISIONAIR including two huge equipments with the RWTH in Aachen and IMMERSIA in INRIA Rennes. Both are non conventional equipments by their size.
VISIONAIR hosted on its VR facilities, a large spectrum of projects centered on new usage of Virtual Reality. 51 projects were completed there. The projects are widely distributed along an axis going from basic, fundamental research on human behavior, in its sensorial, motor and cognitive aspects to the development of new interfaces and declination of Virtual Reality with the motivation to welcome young scientists wanting to test new ideas and push forward the state-of-the-art of Virtual Reality.
On a formal point of view, the outcome of the Work package were in line with the objectives, which was a hard constraint for young facilities (e.g. the engagement of making 20% of its full running time available to other scientists and engineers within Europe and associated countries). However, this also constituted a strong motivation for the platform to develop their activities and the residency of European researchers was a clear benefit for their visibility.
Research activities focused mainly on human behavior in immersive environments, including distant collaboration, ergonomics, robotics and neuroscientific investigations. To select a representative panel of project among 51 actions is hard but let enumerate a few ones:
Sports-men gesture analysis: The test of basket-ball gestures in Rennes or rugby gestures in CRVM measured the gap between the gesture in a virtual environment and the gesture in real space. This was obviously challenging because it was necessary to shot real balls in the CAVE which is a fragile environment and to measure the initial movements to integrate it as initial condition for virtual simulations. It opens indoor training protocols.
Cognitive and psychology studies: perception of the virtual environment is a classical issue in virtual reality which remains a topic of interest. The new step here is that the facilities were expected to provide efficient results for the end-users and thus it was necessary to over-pass virtual reality intrinsic difficulties. Feedback control experiences with very high quality haptic devices were conducted at Poznam University of Technology. In CRVM again Virtual Reality was used to measure attention when driving cars, etc
3D sound experiments: several projects expeted to explore the capacity to immerse persons in 3D sound environement. Some partners invested in new devices to provide this resoure and first test were driven. What is the effect of a 3D sound on the user focuss ? What is the quality of perception of sound directions ? It was measure the threshold of sound perception by a person who perceive a sound without visual feedback and the trick introduced by a fake visual feedback. Such experiences create clear knowledge about the sense hierachy.
Virtual patrimony: the 3D scanner capacities were used in particular with ECN to advance in building virtual copy of old mockup of Liège. New developments were supported to create a complete information system storing all the references of such historical data and innovative virtual environments were produced to be integrated in museums.
Experimentations of co-located activities between several CAVES were organized. One usual challenge is that CAVEs are rarely controlled with a standard software. Moreover the complexity of contents and interactions makes the remote interoperability hard to enable. Sztaky provides a co-space solution named VIRCA, where we can share with standard internet connections a virtual space. Many successful experiences where done to associate CAVES into a VIRCA space: as an example UNIKL connected its CAVE and collaborated with SZTAKI to share a layout organization of a plant with persons in SZTAKI. This capacity was demonstrated in various locations. Other experiences enabled to connect CAVEs by sharing softwares or trying to provide other dedicated connexions.
Obviously all the previous examples do not cover whatever was done by VISIONAIR but they highlight how VISIONAIR opened a wide variety of new use cases in Virtual Reality. We are convinced that it is the key challenge for the renewal of this discipline and the corresponding technology.
• Augmented Collaborative environments including holography
The ACE (Augmented Collaborative Environments) infrastructure (WP7) is built around the need for researchers, to combine their expertise and work power in order to take up contemporary challenges: sustainable development, climate changes, healthcare,novel product development processes.... Great scientific breakthroughs are often invoked in teamwork, and the complexity of scientific problems call for a deep collaboration between scientists.
This observation is true for research inside well defined communities but also for interdisciplinary work. Distinct scientific communities will to fosters their knowledge and competencies on common artefacts that do not belong to one sole scientific field but cross the frontiers of disciplinary work. This work package aims at providing collaborative environments for researchers of various communities, as we claim that direct human interaction are as much valuable as mediated interaction. Visualization must be considered as a key point for mutual understanding and ability to share scientific data and representations, in a synchronous manner.
The equipment proposed in the infrastructure allows multiple co-located persons to visualize and exchange on the same perception (Holographic displays and tables, stereoscopic 3D High Definition projections...) or on multi-form representations. Further more, Augmented Reality (AR) facilities ease the use of mixed visualisation with any possible combinations of real (physical) models and/or digital models. As a result, expert's personal representations of knowledge can be superimposed to form and share new knowledge patterns; Technologies vary from one installation to another, depending of the requirements of users for seamless interactions.
However, visualisation is not enough to support collaboration. Consequently, each installation help concurrent interaction with these numerical representations, thanks to haptic devices, multitouch surfaces, interactive boards or innovative interfaces.
Collaboration may also be provided for distant sites. Hence, interoperability between installations is made possible so that costs, times and environmental impacts of travelling may be drastically reduced, providing that new usage spread of the users community. Video conference systems, available on every installations, reinforce the capacity to interact between distant researchers, sharing vision and applications.
Finally, many facilities can contribute to the materialization of the researchers activities: concept or objects may be prototyped via rapid prototyping facilities, and scientific argues, decision making processes and knowledge expressions can be captured and recorded on long lasting support (audio and video recording facilities). All these equipments rely on (or involve) commercial and lab-made softwares which enhanced the entirely visual collaboration.
Services provided by the infrastructure partners are currently rather engineering sciences oriented, due to the scientific background of the partners. As a consequence candidates researchers to a transnational access will experience cutting edge technologies in the field of product design, virtual manufacturing or decision making process. However, multiple experiences have proven that such devices could serve a wider range of scientific communities.
TNA projects were completed on the ACE platforms. It proves that the collaborative and augmented reality facilities of the infrastructure have achieved a good level of recognition in the European scientific community. New topics were studied extending the range of competencies of the host teams. The new offer provided by the new OPSIMLAB facilities (CRANFIELD) focused on project built around maintenance issues (simulation, training...). Arising during the last year this new facility demonstrated its efficiency and attractiveness with 3 projects.
Here also a wide spectrum of disciplines were welcomed: researchers from ICT, Humanities, Social sciences, Life sciences and biotechnology, mathematics and material sciences. The themes of the projects were also very various, with interests in ergonomic, cultural heritage, museum applications, 3D sketching, healthcare “war rooms”.... Once again, technologies were adapted to suit the needs of the researchers coming from Malta, Italia, Republic of Ireland, Lithuania, Poland...
It's worth to be noticed that partners have joined their forces to allow distant interactions such as in project 156. Such collaboration occurred also in other WPs, illustrating that the collaborative dimension went over the organisational edges of the work-packages.
To highligth a few results among the 21 projects :
The last supper interactive project was supported by Grenoble-INP to integrate 3D haptic device in a story telling application for a new museum presentation of the last supper painted in Milano by Leonardo Da Vinci. Haptic device was successfully connected to a virtual representation of the room where the paint is settled opening new interaction opportunity to better discover this fabulous painting.
Experience to enhance the capacity to better understand 2D technical drawing by projecting a 3D view over the 2D drawings were conducted in Milano. This clear augmented reality demonstration opened new trends for learning or creating new CAD models for industrials practises.
New collaborative practises where developed and experimented to schedule the nurse plan for home care organization. Currently, in the region of Lecco, this work is done by a team of planners who just discuss about the best opportunities when a random event occurs. The introduction of multi-touch table demonstrated its efficiency to enhance tangibility and capacity of collaboration. In the same kind of direction the Design of a Healthcare Analytics War Room to support healthcare data visualization for better problem solving and decision making was supported by University of Twente. This projects highlights the demand of more tangibile and intuitive management of potential crisis scenario.
A more classical application within the scope of augmented reality is the Design of a Virtual Reality Environment for MAintainability TEsts and MAnufacturing Systems Simulations supported by Ucranfield. The potential applications in production systems are arising and the demand to explore the potential deployment of the corresponding technologies is increasing. Here the challenge is to provide mature enough environment to be used in the conditions of a production system.
It was also explored new interaction systems to create 3D parts and to support direct 3D modelling for artists and designers. The results of studies conducted in IPK environments demonstrated the feasibility of the corresponding processes.
Results from joint Research activities
The joint research activities were mainly conducted to improve the quality of service of VISIONAIR respect to TNA projects. Split into three work-packages and more deeply into tasks the organisation of the plan has been reorganised around 20 actions at the end of the first year as anticipated in the description of work.
• WP9
Improvement of human-VR interaction is a key point of many currently performed research projects. Within the VISIONAIR project we would like to conduct our research in this area and what is especially important, we would like to invite researchers from allover the word to visit our laboratories and to make common research. By this way, we will obtain interesting results, establish and improve our cooperation with other partners, which will enable us to form interesting topics for future research.
This work package is concerned with advancing the core methodology to improve the possibility of interaction and collaboration between people through the services to be provided within VISIONAIR.
This will be possible only if we define
1. How we can add interaction capabilities to digital models;
2. How we can improve the possibility of interaction and collaboration on these models;
3. How we can let the distant systems collaborate;
4. How we can propose new methods for new interfaces as holodisplays.
The first point (1) consists in defining new methods for preparing the virtual scene independently of what it is built. This virtual scene preparation needs more and more focus on digital mock-ups that are the core of development of virtual products. This will clearly improve our capacity of access, especially with respect to the WP6 (TNA, Virtual reality) but the developed method should be help toward scientific visualization in our environments (WP4, TNA). The second point (2) aims at allowing effective collaboration on these virtual environments by improving immersion. The proposed methods will be useful to improve usability of our environments on place (WP6, TNA virtual reality) but also between equipment (TNA, WP7, Collaborative environments including holography). The third point (3) aims at building the conditions that allow an effective collaboration by sharing the virtual scene between access points and is clearly dedicated to WP7.
The following actions were conducted in the scope of WP9:
Action 9.1. Integration of multimodal data and kinematic models into VR simulations (GRENOBLE-INP, INRIA, AMPT, CNR, UPATRAS, ECN).
VR and AR facilities can potentially improve product development process when they applied to assembly/disassembly (A/D) simulations, support for assembly and maintenance, ergonomic studies, virtual prototyping in the context of conceptual design and product evaluation. Despite the AR/VR devices’ price becomes accessible for small and medium companies; these technologies are currently rarely used because of the lack of tools to prepare the digital mock-up data for these VR and AR applications. Indeed, the setup of VR simulations requires the transformation and composition of many types of data: the geometric description of the product and its environment, engineering instructions for A/D and maintenance, ergonomic postures, multimodal data for product representation. While these data are often pre-existing, they must be transformed, enriched, and combined (geometric simplifications, combinations of 2D and 3D, …) to enable their use in VR/AR applications. This JRA activity proposed methods and tools that improve the link between multimodal data and VR/AR simulations. Engineering data (CAD, PLM, assembly instructions), as well as multimedia data (images, meshes, posture capturing) are processed with tools and methods to facilitate high-end VR/AR applications.
Action 9.2 Virtual Factory service (CNR, UNIKL, AMPT, UPATRAS, ECN, POLIMI, KTH)
In the industrial manufacturing field, Virtual Reality environments offer good facilities for the assessment of product design solutions according to various points of view (e.g. productivity, working conditions, safety and ergonomics). This is even more important for complex products such as a complete factory. Unfortunately, some limitations exist that prevent the full exploitation of Virtual Reality environments by engineers. Despite the amount of knowledge and know-how contained inside datasets of tools like CADs and PLMs, it is difficult to find the good rule for, on a side, extract the data for building virtual environments and, on the other side, for gathering the digital models that fulfill certain functionalities. PLM/CAD systems provide detailed specification of the conceived product solution, which does not satisfy requirements of VR environment, both in terms of type and complexity of representation. Therefore, these data need to be prepared and adapted to the VR applications. Understanding the exact requirements to fulfil and the needed data transformation and processing sequence requires specific skill and experience that somehow limits a wider adoption of VR facilities to a larger audience.
Research carried out in Action2 was aimed at designing and developing digital
services helping users, on one side, in the transformation and processing
of data from their original CAD/PLM world into representations suitable for
Virtual Reality simulations and, on the other side, on collecting and retrieving
digital models arranged with a specific industrial and logical classification. Focus is on the classification and archival of models for the digital factory design.
The results are integrated in the VISIONAIR VVS (Virtual Visualisation Service) in the form of a manufacturing view of the shape repository and in a repository of workflows.
Action 9.3 Integration and evaluation of interaction potential of Handheld Devices into VR systems (UNIKL, INRIA, UNIVMED, GRENOBLE-INP, UCL, USTUTT, SZTAKI, CNR, AMPT, IPK)
The goal of this Action was to analyse and assess the supportive potential of different interaction techniques for the interaction within virtual environments. One focus was the consideration of Handheld Devices (like smartphones and Tablet-PCs) as interaction device within immersive virtual environments. Therefore multiple prototype applications have been developed to analyse the fundamental applicability and the supportive potential of Handheld Devices as mean of interaction in virtual immersive environments.
As a second focus, investigations have been made how an initial user need
can be supported and satisfied by proper interaction techniques in the best way. The research question was: How to identify interaction techniques which fit to certain user purposes in an ideal way? Therefore interaction techniques need to be described in a non-technical way, outlining the functionality they provide and the benefits which can be achieved for certain applications. By conducting such research, even non-VR experts have the opportunity to understand the characteristics of sophisticated interaction techniques and compare them in the light of well-known applications, from end-user perspective.
For the design and implementation of the new interaction device the research of Action 9.3 and 9.9 was combined for this activity. Within Action 9.9 design and development was being focused to design and develop a device class, which is inexpensive and easy to adapt and extend to different user requirements. The research question addressed here was the integration of the device into virtual environments as well as the evaluation of the design by users in a CAVE. The assessment consists of various applications with different level of interaction. Out of this assessment various improvements and features have been added to the design.
Action 9.4 Annotation capacities for asynchronous collaboration (RWTH, INRIA, UNIVMED, ECN, CNR)
For the creation of an annotation system that facilitates the asynchronous collaboration in VR environments, the partners involved in this joint research activity identified three main areas that need to be addressed: storage & retrieval of annotation data, creation of annotations, and presentation of annotation data. The development of a comprehensive annotation system that covers all three aspects is a challenging task that goes beyond what can be achieved from within a VISIONAIR JRA only. As such, we (partner RWTH) have combined work from multiple research projects funded at other levels, making efforts to develop solutions for each area. Within VISIONAIR a prototypical annotation system based on this work is now available.
ECN has been working on reverse-engineering, 3D visualization and knowledge management for cultural heritage. We started a project in 2008 with a museum to apply theoretical research on museum collection objects. In 2011, we started a collaboration with Belgian researchers to confront our ideas and methodologies. We published two articles in order to set up a new methodology for cultural heritage objects capitalization. It goes through 3D digitization, information modeling and visualization.
Action 9.5 Real time 3D Modeling for Interaction and Presence (INRIA, UPATRAS, UCL)
3D modelling sensors relying on cameras sometimes extended with depth sensors enable to compute in real time a 3D clone of the user. The popularity of such approaches is growing rapidly, the Microsoft Kinect probably be the most visible example, this device enabling marker-less interaction in games. In VR environments the full potential of such devices remains to be explored. The geometry extracted opens the way to marker-less interactions, while, once complemented with photometric data, it enables to build a virtual clone of the user usable for enforcing user’s presence in the virtual environment. The goals of this action is to explore the potential of real-time 3D modelling for improving the user’s ability to interact with the VR environment, to collaborate locally and to enforce its telepresence. In addition, the potential for use of such interaction devices in holographic applications, will be explored and new interaction metaphors will be developed for facilitating local collaboration and human-centred interaction.
Action 9.6 Action to awareness in 3D environments (SZTAKI, USTUTT)
Awareness provides a way to get informed or understand the situation regarding entities of interest. Awareness information generation deals with deciding what context and raw information is necessary to create awareness of a specific situation or state. For example, to generate data that describe the fact that a specific user is “actively working“ requires not only the “user presence” context information but also historical data about what he has done. One important research topic in this area is to figure out what information and in what details is necessary or useful to be presented for the users. The importance of awareness information increases
in the case of remote collaboration, where the better understanding of the remote partners’ status might be essential for the efficient collaboration.
MTA SZTAKI proposes a generic awareness handling system suitable for Virtual environments. This proposed system consists of a central message repository
and distribution server, several data collector agents and several data
consumer agents. The system is suitable for adding and visualizing
contextual data in a given collaboration environment.
Virtual reality installations usually are custom designed to fit the target application based on experience of users or companies. The design and capability essentially affects usability, a CAVE might not be used as an static image viewer or a 3D Desktop VR on a small display might not gain full user immersion for instance. From this point of view, to assess and to evaluate hardware and projection setup design corresponding target applications a set of criteria for grading utilization and interaction of the user is needed and has been proposed within this research action.
Various tracking data from virtual environments regarding user interaction in specific applications or application classes respectively is being harvested and stored in a data base for qualitative comparison between hardware setups running different applications. This will improve future design matching specific applications to potentially increase immersion and awareness in 3D environments.
Action 9.7 First versus third person perspective for large scale navigation (UNIVMED, AMPT)
During the project, we conducted a number of experimental studies, in which we manipulated a number of factors, while measuring cybersickness symptoms, using the standard "simulator sickness questionnaire" (SSQ, Kennedy et al., 1993). Participants were immersed inside a 4-sided cave system, and submitted to a spatial retrieval task, in which they had to find a place (goal) in a large complex environment.
In a first experiment, using first-person navigation with a flystick, we tested the effect of sensorial aids, a spatialized sound or guiding arrows on the ground, attracting the user toward the goal of the navigation task. Results revealed that sensorial aids tended to impact negatively spatial learning. Moreover, subjects reported significant levels of cybersickness. In a second experiment, we tested whether such
negative effects could be due to poorly controlled rotational motion during simulated self-motion. Subjects used a gamepad, in which rotational and translational displacements were independently controlled by two joysticks. Furthermore,
we tested first versus third-person (avatar) navigation conditions. In the
second condition, cybersickness tended to be lower. However, large
interindividual differences made the difference non significant.
A third experiment was realized, in which we simplified the environment. The participants simply had to reach the exit of a one-way maze. To do so, they were given three locomotion interfaces, across three different test sessions. The first interface was an Xbox controller, on which two minijoysticks independently controlled forward motion and horizontal rotation. The other two interfaces consisted in custom adaptations of a "redirected walking" algorithm: users had to physically turn their body toward the direction they wanted to go and a slow counter rotation was then applied to the virtual environment, such that they were redirected to face the cave's front wall. The general hypothesis was that this would prevent fast visual rotations, hence potentially reducing cybersickness. In the second interface, users had to walk in place, and passive captors on their knees were used to calculate locomotion (forward) speed. In the third interface, the Xbox controller was used for forward speed along with the "redirected" algorithm. The comparison between these last two interfaces was meant to measure if active stepping would reduce cybersickness. Results from this study show that 1) redirected walking is easily accepted and intuitively used by all participants; 2) that walking in place is reported as tiring and results in greater speed variability than the other two interfaces; 3) that, on average, all interfaces result in self-reports of cybersickness, even if exposure time was quite short, always under 10 minutes. Finally, it appears that users having no experience playing video games exhibit higher scores of cybersickness than "gamers", suggesting that, besides interface properties, individuals factors come into play.
Another study related to navigation investigated the use of natural language and user gesture for navigating in real scale virtual mock-up of a building. Two sets of interfaces were introduced: 1) interaction devices, 2) natural language (speech processing) and user gesture. The survey on this system using subjective evaluation (Simulator Sickness Questionnaire, SSQ) and objective measurements (Center of Gravity, COG) showed that using natural language and gesture-based interfaces induced less cyber-sickness, as compared to device-based interfaces. Therefore, these results suggest that gesture-based interfaces might be more efficient than device-based interfaces.
Finally, a study related to driving simulation has been conducted. The added value of vibrations during driving simulation was studied. Current dynamic driving simulators induce simulator sickness and it is still difficult for the driver to project himself in the virtual reality, due to incoherent multi-sensory stimulation and perception. To know the effect of vibrations on a subject, the effect of a whole-body vibration must be defined, as the sources of vibration in a car cabin. After determining all the parameters, a formula was proposed, to produce vibrations as a function of the car state, the road and the boundary conditions. An experimental study was realized, with nine subjects. In order to do this experimentation, three actuators were installed inside the cabin of the car driving simulator. Data analysis is ongoing.
Action 9.8 Deployment and analysis of collaborative software (INRIA, UCL, UNIKL, SZTAKI)
The Joint-Research-Activity deals with the implementation of VR distance collaboration tools and their usage in the scope of multiple application fields. The Research Question was: How can distance collaboration tools be deployed to different VR systems and beneficially applied for different research topics?
Thereby the development of new systems that allow the coupling of different immersive VR systems was not the objective. The goal was to test out the capabilities of existing distance collaboration tools (in detail COLLAVIZ and VirCa) and assess them against each other, based on certain use-cases. The first topic of interest was how complex the deployment and adaptation of academic software prototypes on new VR systems is. Challenges in the installation and usage of software prototypes to Partners infrastructure who have not been involved into the development of the tools itself were analyzed. The second topic of interest was if the analyzed distance collaboration tools can completely support the dislocated work for certain use cases. Thereby interaction, visualization and collaboration functionalities are crucial. Therefore two typical use cases (collaborative visualization of seism data and a factory planning project) have been developed and executed by the help of the software tools.
Out of this assessment different distance collaboration tool features have been identified and assigned to typical use-case demands. Criteria to improve the future development have been proposed, as well as advices for the usage of the distance collaboration tools could been given.
Action 9.9 Interactive devices and tangible interfaces (PUT, IPK, UNIKL, UNIVMED, USTUTT, SZTAKI).
The applied today system for human communication to VR systems include only: different human tracking systems, joysticks and haptic devices. In VR systems there is still a need for development of new human-VR environment communication methods. The aim of this action was to support the development of highly interactive physically enriched VR systems and to ease the integration of various types of interaction devices, physical (tangible) objects and tracking technologies.
Furthermore a haptic manipulator (haptic joystick) was developed by PUT and integrated into an stereoscopic projection system and a tracking system in order to study the integration of various kinds of interaction devices, physical objects and tracking technologies within the VISIONAIR JRA and TNA projects. The built haptic device (light robot) enabled the improvement of the human - immersive VR communication and interaction. The main task of the haptic device was to move its ending arm to the position in which the operator's hand may touch the visualized object. The haptic device expanded the immersive VR system with tactile feedback.
In the project, the Fraunhofer IPK developed a software framework called Tangible User Interface (TUI) framework which provides a high-level interface and duplex access to important properties of interaction devices, interactive objects, and interaction tools.
The action was also focused on enabling haptic feedback in immersive case scenarios. To do so, a new haptic device was constructed and an appropriate VR software was developed at PUT.
Furthermore, after deep analysis it was decided to integrate the newly developed by PUT haptic device with the TUI framework worked by IPK. Initial tests proved that such an integration was possible by using the UDP network data exchange protocol.
Action 9.10 Research into Cognitive Infocommunications (SZTAKI)
CogInfoCom proposes a new and unified conceptual approach in which the process of merging is derived from the theoretically unified concept of different levels of cognitive capabilities co-existing in the information space (irrespective of whether they are natural or artificial capabilities, and whether they are individual capabilities or capabilities which emerge from a cloud of artificial and/or biological components). This derivation extends to various aspects of the merging process, such as interactions and communication, as well as increasingly flexible interfacing between networks of living beings and artificial cognitive systems etc
SZTAKI is managing the CogInfocom conference but many partners (Grenoble-INP, INRIA, UNIKL, etc) were participating to the action with the organisation of special session centred around the VISIONAIR activities.
• Advanced Technology for Interaction and Collaboration (WP10)
The following actions have been listed in accordance of the tasks described in the Dow:
Task 10.1: Holography technology improvements and technology evaluation of high resolution curved tiled display
In this task the holographic technique used in VISIONAIR is improved. The work has been concentrated on developing the technique so it can be used in a collaborative environment - both locally, with multiple users at one display, and remotely, connected to other displays which may be of another kind.
The Holotable is an autostereoscopic display created at KTH. Its previous prototypes could only be used by one person at a time and had, hence, limited usage as a collaboration tool. It has now been developed to a multiuser display in the form a table where two users sit on opposite sides and can see a 3D image floating above the table surface. In the previous prototypes one linear array of projectors provides a viewing zone for one person or possibly several persons sitting side by side. For the new version the design idea was to add one more array of projectors, oriented parallel and displaced perpendicularly to the first array, to provide additional viewing zones on the opposite side of a table-top display. These additional viewing zones can show images independently of the first viewing zone, e.g. the opposite side of a three-dimensional scene. The HOE has to be modified somehow so it can show the viewing zones from one array of projectors on one side and viewing zones from the other array on the opposite side without mixing the two. We have investigated three methods for achieving this. We have also modified the software accordingly. The current prototype uses an HOE of size 30 x 40 cm. Our single-sided display has a 60 x 80 cm display area, and we expect that we can make a double-sided display in that format as well. We use four LED-projectors on each side. The projectors are driven by two ATI Radeon HD7970 graphics cards mounted in the same computer. A maximum of six projectors/screens can be connected to each graphics card. The image quality is virtually the same as for the single-sided prototype. The view-zone size and viewing angle for each viewer can be the same as for a single-sided display. We have also shown that it is possible to make three and four-sided holotables.
Task 10.2 Create an Interoperability Environment among Distributed Platforms
This task concentrates on network transport and protocols, streaming and codec technologies and on developing comprehensive vision for WP10 work scenarios. The scenarios form the base for evaluating appropriate state of the art technologies regarding networked communication, collaboration and interaction. They also define the demands on contenIn this action the work was focused on software development and establishing network environment for remote collaboration in order to enable the scenario defined as a research question. To achieve this goal it was necessary to fulfil several requirements. Significant part of work has taken adapting VITRALL, a distributed web based visualization system enabling a real-time visualization of a complex 3D content using remote distributed servers equipped with modern multi-GPU solutions, to support functionality required for proposed scenario. It included: 1. Integration with JPEG2000 streaming platform used by two VISIONAIR partners: University of Essex (now the team in Bristol) and PSNC as well as external users, such as SURFNet and WAAG Society from Netherlands, CRC in Canada or ITAI PAS in Poland, 2. Implementation of new module supporting strict requirements of video streams synchronization for stereoscopic and holographic displays, 3. Implementation of new JPEG client in order to support multistream displays - UHD 3D in PSNC and Holographic table in KTH, 4. Enhancement by functionality of reading remotely located 3D models, to enable sharing of models between rendering and modelling systems. 5. Implementation of new module for tracking and uploading modification done to the model by external software – to allow integration of the system with arbitrary modelling software (Blender, 3ds Max, etc.) Platform integration activities focused also on the network streaming solutions, advanced streaming technologies required for transmission of ultra-high-resolution multimedia streams has been tested. The main aim of network experiments was to verify integration of different platforms and identify possible issues. It was also necessary to check the throughput and latency to assure high quality and comfort of on-line cooperation of designers. Tests were conducted gradually to validate all elements of the system and eliminate limiting factors. Additionally, HPDMNet (over GLIF physical infrastructure) and GEANT connections (direct or through OpenVPN) were established in order to achieve research goals of this action. Finally after fulfilling all conditions necessary to successfully realise proposed scenario, it was shown on VISIONAIR Open Forum 2014 in Poznan. The demonstration has shown remote collaboration in modification, rendering and visualization of 3D models. In this performance was used the scenario, where a group of people can work on one 3D model and perceive the same virtual object from different perspectives and views using different kind of displays such as holographic table or UHD 3D displays. The scenario presented during the Open Forum conference involved designing a children’s playground. In this process there were three sides located in different places and with different visualization capabilities. Each of them had the opportunity to view created model independently, discuss on-line all changes and make decisions about further designing actions. One of the parties involved into demonstration was a 3D model designer located in Barcelona. He used Blender modelling software for editing 3D model and making changes proposed by the other participants. 3D models were shared across the network, so all participants were able to see effects of his work immediately. The second person was sitting in front of holographic table in Stockholm. VITRALL server located in Poznan was rendering 3 different views for 3 projectors serving the holographic display. Holographic table operator could move and rotate the 3D model in order to see it from all sides. The third side was an audience on Open Forum in Poznan. On 4K 3D screen there were projected all views of models as well as two remote sides via videoconference. Demonstration allowed presenting in an accessible way capabilities of VITRALL. In this live design and discussion process the playground project was being modified by inserting additional items (swings, ladders, etc.) and changing their position and appearance. The show has generated a lot of attention, as easily applicable for real-world scenarios, arousing many questions after it.
Task 10.3: Create technologies for collaborative communication and interaction between networked holographic table displays
The partners will collect information about, implement and evaluates interaction technologies appropriate for holographic table autostereoscopic displays and similar displays. As there are very few holographic tables around there is a need to develop new interaction schemes and tailor requirements in accordance with developed application scenarios.
The work focused on the development of a solution which offers the possibility to have live remote view of virtual and dynamic objects. These objects can be a combination between physically present objects and virtual objects, and should be live viewed from one or more remote locations. To experience and research the possibilities within this focus multiple project have been executed and tested.
VITRALL
NICE
Distance collaboration support environment
3D printed head mounted devices for communication and discussion
Remote lab management
Virtual Factory
All solutions allow for viewing objects from one or multiple locations. The differences can be found in the number of participants, the combination between real and virtual objects, sharing digital environments and controlling remote locations.
The work described in this document focused on enabling and experiencing live remote view of virtual and dynamic objects. The interaction between viewing real and virtual objects, in combination with interaction possibilities and ability to do this over a large distance, was examined and tested in multiple settings.
• Conceptual Modeling, Assessment, Metaphors and Social Impact (WP11)
During Period 3, the main activity of WP11 was to finalize the actions which have been defined at the end of first year within deliverable “Visionair-Deliverable 11.7” : Detailed Research Program - Conceptual Modelling. This deliverable was reorganising the tasks of the Description of Work into six main actions. During period 3, the technical work was finalized and reported on the VISIONAIR site as for WP9 and WP11. WP11 was concerned by the following six actions:
Action A11.1: Modeling the user engagement with VISIONAIR: An Object Process Modelling representation of VISIONAIR TNA process was constructed and mapped in a 3D space
The action consisted of two steps corresponding to the two research questions mentioned above.
Modeling of TNA process using the OPM framework: A team of students from Technion was in charge to translate an oral definition of the process presented by the VISIONAIR scientific coordinator into the OPM framework. We designed a detailed conceptual model of the TNA process. This task presented an opportunity to deeply analyse the TNA process and get a better overall system-wide comprehension of the TNA process sub-tasks.
Instance representations: In this original work we visualized the system after observing that even though the conceptual model is quite clear, the visualisation of more than 200 projects make the perception of instances very hard. To alleviate this problem, the conceptual model was converted into 3D representation that provides for direct and tangible perception of projects.
The main proposition is to develop a model that present instantiation of OPM. We tested this approach on the instantiation of the TNAs OPM model. This model was created during the first JRA year of VISIONAIR in collaboration between Technion and Grenoble-INP. The OPM model is basically a 2D graph. The main idea is to use the third dimension to visualize instances. The OPM models has the ability to represent on the same graph both the processes and the objects handled by the processes with their various states which makes a clear understanding of the potential evolution of the objects.
The model was slightly updated and reproduced in a 3D environment. Project, Review Committee, Host Facility and Research Group are the four main objects participating to the main process. In 2D the arrows clearly highlight the activities that can change an object state into another state. But if we create an instantiation diagram then the corresponding graph includes the 200 projects, all the review committee members, the 29 VISIONAIR host facilities and about 200 external research groups which submit projects. This graph will not be readable.
Thus OPM is perfect to get an abstract view of the overall process, but it does not provide a correct solution for instance visualisation. We extended the view by extruding the OPM 2D graph in the third dimension by placing in front of every object class its corresponding instances. The system was also enriched with interaction capabilities to support selection of objects and states to be visualized.We tested this new representation of the instance model as follows:
on an holographic screen we found that the resolution was too low to enable reading the names of the instances;
On a HD stereoscopic display we found that 3D perception enables a very good perception and navigation (Figure 4);
On a 6K 2D display (3x3 HD display matrix) we found that the quality of character strings is much higher and is a real improvement, but being 2D it does not provide the added value of 3D perception.
In conclusion, this 3D representation may be a real added value for huge data set perception. As soon as text is visualized in ultra-high resolution we can expect to be able to read this texts. Sterescopy is a clear advantage to perceive relative positions in the 3D scenes. The use of such representation should be adequate with UHD 3D stereoscopy displays.
Action A11.2: User requirements and expectations capture – A roadmap for constructing Synthetic Environments in VISIONAIR.
A TNA can only be efficient and effective for both parties if it is preceded by adequate preparation. Aligning requirements and assessing resource usage
and working methods are essential here. Adequacy of these preparatory processes is not obvious, as the stakeholders are geographically scattered all over Europe and have their own backgrounds and interests. Therefore, supporting the underlying deliberations can yield huge benefits in preparing a TNA.
A TNA is not a unique phenomenon. Many situations exist in which stakeholders from different backgrounds collaborate to achieve a specified and predefined conjoint goal. In product development, so called synthetic environments (SEs) are an example of a setting in which such cooperation takes place. Given the way in which SEs are developed, established, used and evaluated, there is clear similarity between these SEs and TNA's. Consequently, the preparatory processes of SEs and TNAs are also similar, and the tools and working methods used in preparing and running SEs apply to TNAs as well.
Action A11.3: Mapping user requirements to VISIONAIR resources through enhancement of the resource e-map: A tool was developed in order to suggest the most appropriate combination of the VISIONAIR infrastructure services that could best fulfil the user requirements and expectations.
This action aimed at providing a tool for the external user enabling them to assess and evaluate, through the web-based E-map tool, what facility would satisfy his/her requirements and to what extent. The requirements on visualisation, interaction, associate and software facility resulted from the E-map ontology were used for the extraction of information such as display and tracking devices, manufacturing machines and software tools, respectively. For each user type, a different route was identified for capturing the necessary user input, so as for the system that follows to ultimately provide the infrastructure classification based on that input. The expert users are able to identify more easily the specific requirements. For example, an expert user might be interested in having force or tactile feedback, while a novice user with the same requirements is questioned whether advanced hand motion operations are in order.
The algorithm which is used for the assessment of the infrastructures based on the user's requirements is by ranking the available alternatives. The alternative with the highest utility is defined as the best alternative. The utility of an alternative is calculated by normalizing its requirement values. In the current phase, all the criteria are weighted equally. The decision making for the available alternatives can be formalized in a decision matrix, where the rows represent the alternatives and the columns represent the criteria/user requirements. By using the appropriate equations the infrastructures ranking is produced.
Action A11.4: Framework for modeling and assessing collaboration process within VISIONAIR: A questionnaire provided a qualitative map of the interactions between collaborators.
This action aimed to identify the enablers and barriers that will improve or hinder collaboration activities throughout the life cycle of the VISIONAIR reviewing process. The methodology is comprised of generating a list of collaboration enablers and brakes/barriers derived from previous work and literature review. We propose a four-step process to ensure collaboration assessment:
A questionnaire is first administered to a small number of people involved in the review process to extract the most important factors which can be used to describe collaboration activity and identify the main enablers and brakes for collaboration. This questionnaire needs some training to be filled. It is thus dedicated to a limited set of representative persons and projects: filling the questionnaire, a selection of the Patel's keywords must be done. By analysing the results it was possible to identify which keywords were used and which have no sense for the given context.
A second questionnaire is administered using the most important factors identified in step 1. This questionnaire was direct and easy to answer for any person involved in the assessed collaboration. It lead to the evaluation of a restricted list of factors. This questionnaire is dedicated to be widely filled. It will ask for each phase of the assessed process to define which collaborations were used and the main enablers and barriers within this collaboration.
Analysis of results: the second questionnaire indeed provided a qualitative map of the interactions between collaborators. Then some social network assessment techniques may be used to interpret collaboration capabilities of the organisation or system which is under focus. To validate this method and adjust the questionnaire we checked it first on a well known collaborative process. It has been a good way to finalize the process.
The second questionnaire was tested on a single TNA project and enabled producing a collaboration graph where the connections between roles are highlithed and where we can see the positive and brake factors with respect to collaboration. The main conclusion is that such a method is useful and we proved it can be applied, but it needs to be refined. The information acquisition should be made more automated so collaborators can tell for every collaboration the impediments and facilitators of the collaborative work in a simple way.
Action A11.5: Conceptual Model for four visualization applications: In this JRA, we used physically small but complex system to demonstrate how OPM's Object-Process Methodology (ISO 19450 Standard) enables us to effectively communicate complex business models.
The aim of the project was to develop affordable servitisation business models based on the GE Vscan ultrasound for use in primary care. Being able to use the Vscan in this sector would permit to improve the patient experience and reduce costs for the protagonists of the primary care. Several objectives were set for this project:
Review current processes for image capture between primary care and hospitals;
Develop a series of visualized servitization models;
Develop cost/benefit analysis for the range of possible business and service models;
Validate these models against current processes with end-users; and
Explore the application of potential visualization technologies.
First, product service system (PSS) models were developed in order to demonstrate the patient's journey. Experts validated the proposed business models. For visualizing, understanding, and communicating these business models, visualisations were developed using OPM models and Vivid OPM. The first one permits to reduce the journey of the patient to one visit to the general practitioner. The second model validated the business model, based on the services provided by a radiographer who visits the surgery on a weekly basis. The third business model is an extension of the second approach. However it requires a diagnostic radiographer qualified enough to analyse the images. Finally, in the fourth model, a mobile radiographer visits the patients at home. All four models were validated to be feasible, so a healthcare organization can pick what best fits its policy service model. Quantifying the cost of each model will enable finer optimisation of the model to be selected. In addition to its value for promoting affordable healthcare services, this JRA serves as a demonstrator of how OPM Object-Process Methodology (ISO 19450 Standard) can be utilised to effectively evaluate visualization technologies in general and those developed under VSIONAIR in particular. In this project, VISIONAIR has made notable contributions to healthcare in the public sector.
The wide array of research domains and applications related to visualization is a clear demonstration of the success of VISIONAIR, to be a focus of knowledge in all aspects of visualization across Europe and a world-class center for visualisation technologies and research. VISIONAIR is continuing its journey in advancing the state-of-the art in visualization and dissemination of visualization-related knowledge to all the interested European researchers.
Action A11.6: Model based approach for developing collaborative environments: This work presented (1) ergonomic issues (2) awareness issues- (3) collaborative environment for real world applications.
We aimed at developing a knowledge base that can be used to understand the complexity in developing collaborative environments for multi-functional teams. This work presented three aspects of knowledge relates to this issue:
(1) Ergonomic issues - most of collaborative virtual environments' design approaches are driven by technological constraints. The disadvantage of such approaches is that they produce virtual environments often far away from users' needs regarding collaborative activities.
(2) Awareness issues- there are known methods for evaluation of the awareness of users, which are more connected to the usability of the whole system than to the awareness subsystem. Quantitative measures are hard to execute, and the most widely applied methodology for evaluating user awareness of processes are questionnaires asking the users about their feelings. Using this methodology researchers were able to compare different awareness implementations, but not their effectiveness. Beside the evaluation of the effect of the applied awareness information via questionnaires, researchers could measure the productivity of the participants directly. From these measures they were able to collect some quantitative values about the effects of using different awareness types in their use case. Performance indicators were also important, but this methodology is not always applicable since it requires several independent target specific measures.
(3) Collaborative environment for real world applications - the proposed urban planning system identified the tools required to support the stakeholders' activities in the TF programme. These tools allow individual stakeholders to better communicate and share data with other team members as well as to support their own work. Urban information frameworks can allow the aggregation of various dispersed and distributed urban datasets from various organizational stakeholders to create a virtual prototype for an urban area. Modelling the TF as a case study assisted to define the objective of each organization involved in the program as well as the user activities required to build a sustainable community in order to combine data sets and enhance collaboration amongst the multi agents involved in the program.
Potential Impact:
The impacts of VISIONAIR are in multiple directions:
Fundamental research was directly impacted by the proposal of new resources “easy available” with technical support. Most technologies deployed by VISIONAIR are high end technologies, often prototype systems with a single existing configuration. Then investing and maintaining these solutions remains a research speciality by itself which is usually partially exploited with a limited number of research directions. The capacity for researchers to get benefits of these technologies without investment and maintenance issue was just an opening at the international level. Nowhere researchers who could need these techniques can access such a wide panel of resources without investing a big part of their activity in the engineering work. Here VISIONAIR radically changed this situation. The ramp-up in trans-national access projects demonstrate that it was not easy three years ago for an external researcher to understand the benefit of accessing these technologies for their own researches. But the first projects, first success stories create a kind of buzz. At the last general assembly in Rennes, a biologist already involved in visualization made evidence that he was practising these technologies as a very good handyman but that accessing Virtual Reality CAVE within VISIONAIR demonstrated the capacity to radically boost his research. He told that one day in such an environment could lead to one publication where with usual techniques several months where necessary for the same result.
Innovation in processes and practises is another direction where VISIONAIR demonstrated a clear impact. Technologies are existing and evolving very quickly. Partners of VISIONAIR participate to the development of technologies but a major contribution here was the test and validation of processes using the visualization and interaction technologies. We often mention the demonstration in ICT Vilnius where a dancer located in Grenoble danced on a music improvised by players located in Barcelona, Poznan and Vilnius. A 3D avatar of the dancer was produced in realtime and projected in a virtual scene surrounds by streams of the players. The virtual scene was projected in 3D to the audience in Vilnius. This is not science fiction nor deep fundamental research. Technologies exist but the strength of VISIONAIR was to be able to cluster the technological resources and to organize them for this single and amazing event. Such a demonstration is the proof that this process is really feasible. It brings the idea to a real innovation quite ready to go to the market place. The VISIONAIR individuals who were on the VISIONAIR booth at ICT in Vilnius made evidence of the impact of the demonstration for the witnesses. Many other TNA projects were submitted afterwards to repeat partially or fully or to overpass this demonstration. It is the same when we consider surgery tele-operation tested and awarded via the HYPERMED project. It is the same when we invite sportsmen to analyse gestures in a virtual reality CAVE. Projects to visually monitor Police or Firemen activity are anticipating the incoming concept of smart cities. VISIONAIR demonstrates the feasibility of the processes and open the way to the deployment of the technologies.
Opening facilities was also a matter to open the mind of VISIONAIR partners. It is a deep impact indeed to enter in a process where we must share the technology we developed for a very specific usage and thus to adapt our processes for other applications. First it ensured a more complete usage of expensive facilities which were sometimes poorly used. Secondly it enforced the creation of new interoperability process and it could be easily demonstrated that every partner passed a big step for organising projects. It thus also increase the maturity of partners respect to the technologies they maintain leading to a combinatory explosion of activities. It also highligth the need to create new integration platforms at other levels. VISIONAIR was a group of 24 partners and should integrate much more facilities. The interconnection of these facilities for a better research and civil usage is inherent of the development of the technology itself. That means that VISIONAIR his a “has been” model that should be widely extended in Europe. As it will be discuss just after we fully share the concept of community ring developed in the SPECIFI project (http://www.specifi.eu/) where a VISIOANAIR like facilty must play a specific role.
Industrial, economic and citizen impacts: as a scientific infrastructure VISIONAIR did not include non academic partners. But indeed a first link was clearly and naturally established with visualization and interaction device providers. Then links came with end-user and once again by the need to innovate and demontrate the possible deployment of specific use-cases. Almost 10 partners from VISIONAIR are directly in connection with industrial applications because their research field is the development of tools for production systems. Another set of partners are more directly connected to healthcare activities and the research conducted within VISIONAIR will be deployed in this sector. Not least a few partners were really involved within city applications: connections between artists, interdisciplinary collaboration, etc. Even if the duration of a project in VISIONAIR cannot directly lead to a patent, the infrastructure indirectly participate to innovation in all these sectors.
European attractiveness: VISIONAIR was conducted with a scientific board. We invited recognized scientists in this board, from the US, from Brazil, England and Germany. And obviously we had much more contacts with scientist and engineers outside Europe. We clearly share the expectation of an infrastructure model delivering research resources to all the scientific community. While network communication system are widely developed worldwide, while disciplinary infrastructure are created, we also need intermediary transversal support for the middle ware between hard communication and human expertise.
A Key Innovation Enabler Visualisation as a key enabler assists in dealing with typically complex challenges i.e. to first understand and subsequently manage factors, relationships and interactions. Complex challenges (e.g. environment; sustainable healthcare; economic prosperity) are likely to be relevant to different disciplines and cross national boundaries. Visualisation has a role to play from two perspectives:
(1) communication across participants in different locations, and
(2) to interpret and analyse data.
The findings reported in this final version of this deliverable, are based heavily on the internal and external recommendations gathered in the context of Task 1.3 “Internal and external interactions”. Internal interactions recognise specific existing technologies and the areas that they have been applied. At a high level the areas are classified based on industry sectors, whilst further analysis is required to be conducted in order understand the challenges. On the other hand, external interactions help to understand the challenges faced across industries and position correctly the current issues that need to be addressed.
Priorities for European Innovation: nowadays, most scientists usually practice in 2D visualization, which covers their main needs, therefore VISIONAIR needs to identify:
a) The real need for 3D applications
and
b) The communities that really need advanced visualization technologies.
Research priorities for advanced visualization should indicate the strategic areas in which the long-term vision of VISIONAIR should target and the directions it should follow within each of these areas.
Strategic areas: VISIONAIR has identified the following strategic areas for advanced visualization research:
Manufacturing: visualising data about the life cycle of engine/part/system both at the early design stage and operation. Illustrating, dynamically, forecasted data about time to failure and utilisation.
Design: accelerate product development, time-to-market and reduce prototype development costs by using virtual prototyping techniques.
Computer supported collaborative work: Open universities are becoming increasingly popular for people to continuously educate themselves. This means that individuals do not need to attend lectures and can utilise provided material or online courses as it suits them. Universities dispersed in different locations are collaborating and in order to deliver targeted outcomes. Visualisation has a role to influence the level of collaboration and the quality of the outcomes.
Cognition: Complex user studies that involve the cognitive analysis of humans can be undertaken in advanced visualization facilities.
Medicine: Governments around the world are challenged to reduce the cost of delivering healthcare services to patients. A trend has emerged that promotes preventative measures while surgeries are increasingly attracting interest to facilitate this. However, surgeries tend to be less equipped and overall may be less skilful to conduct various forms of examinations that are made in hospitals. A major characteristic of these examinations is to visualise the health of the patient and technologies that facilitate these in an affordable, user friendly (e.g. less skill requirements) manner. There are also opportunities to provide very detailed and accurate images of patients in an operation for a GP (general practitioner) in a distant location and to share data about health conditions in a secured manner.
Ergonomy: ergonomic studies for products and processes require complex tracking systems and advanced algorithms for measuring the user’s movements and behaviour. Ergonomy plays an important role in the design of many products and processes.
Biology, Chemistry and Nature observations: complex organisms and mechanisms of the nature cannot be replicated in a control environment without significant cost and time. Environment is a cross industry aspect, where there is a need to be able to compare and contrast the impact on environment from different kinds of production and service provision models.
Physics simulation: many applications require the simulation of the physics phenomena taking place during their observation. Visualizing the behaviour of materials under stress can provide inside in to their failure mechanisms etc.
Arts, multimedia and patrimony inheritance: museums, architectural design of urban spaces and other applications can be benefited from interactive and immersive simulation environments.
Future advanced visualization research and development follows certain directions based on the current trends and challenges. Therefore, it is important to be established which directions are relevant for each strategic area, so that it can be followed through a suitable business model.
By the end we are convinced that VISIONAIR fully participated to the visility European Reasearch Area and that an extension of VISIONAIR can continue with this role.
Main dissemination activities and the exploitation of results
Dissemination was mainly organized in the work-package 2 but not only. A lot of activities were directly driven through trans-national access contacts or peer to peer contacts on the initiative of specific partners. The strategy has been first to build a wide awareness about VISIONAIR but it was not enough. The matter was to convinced external users that it was not impossible to access the facility as soon as a reasonable project was proposed without out decreasing the quality of accepted projects. Then a second strategy step was to connect VISIONAIR to other facilities and projects. We enter in deep contacts and actions until with TNA projects with the SPECIFI initiative opening a huge area of activity in the direction of smart cities and collaboration rings. We enter in discussion whith PRACE about the interest of Visualization and Interaction within HPC. We enter in connection with observatories, and with scientific societies to create better links with communities.
A specific effort was conducted to participate within conferences and events concerning disciplines which were not represented in our TNA accepted projects improving the interdisciplinary activity of VISIONAIR. The JRAs also participated directly to the creation and the promotion of knowledge and obviously every publication was expected to cite VISIONAIR. The same for Trans-National access projects who led to a lot of common publications.
But indeed WP 2 dedicated to external outreach was the official organization of dissemination. The following sub-sections outline the targets for the specific tasks.
Task 2.1 - Web Portal and newsletter
This task aimed to sustain a web portal and newsletter. The web portal involved updating the website with up to date information about proposal submission, areas of interest, example TNAs and news items (e.g. EU deliverables, newsletters, and VISIONAIR events). The web portal has been fully functional, and supports with achieving some key targets such as building a dynamic relationship with external communities, attracting new proposals, disseminating outcomes of VISIONAIR (e.g. deliverables), promoting events, attracting institutes to the Partners Club and building a visualization community. In parallel, the intranet, which enables internal communication, has also been fully functional and has been successful at engaging over 90 colleagues. Task 2.1 has achieved the targets set out for the project. In the last period an area that was a significantly enhanced in the web portal was ‘Browse VISIONAIR Resources’.
Task 2.2 - Attending Trade exhibitions and international events:
The task 2.2 developed the strategy on joint external outreach at international events for VISIONAIR. As defined in the developed process models, the approval and information of Partners attendance towards international events was handled by the task 2.2 involved Partners. Advancing with new project results, the focus of attended international events was increasingly on the dissemination of JRA results and TNA projects. By sharing the experiences gathered during several TNA projects, external audiences have been introduced in a lively way to the opportunities, facilities and know-how offered by VISIONAIR. The presentation of JRA results at international events increased the visibility of VISIONAIR at international scientific level. Hence the presentation focus was not only the sheer research results, but also the activities regarding networking between partners and building up a shared infrastructure.
The successful cooperation between the VISIONAIR partners and external researchers about TNAS lead to attendances at events which directly showcased the capabilities that the TNA model offered. As direct result out of TNA projects joint papers and attendances to international events were realised, whilst incorporating VISIONAIR partners and the external TNA guests.
In addition to the regularly attended international events of the Partners (e.g. conferences to introduce TNA and JRA results) task 2.2 worked on the attendance towards key events on which VISIONAIR contributes to larger extend. These key events are determined by demonstration activities (such as demos, booths…) to highlight the capabilities offered by the consortium. By addressing such events the visibility of the project could be enlarged.
In the last year 48 international events have been attended by VISIONAIR Partners, to publish results (TNA results, JRA results and additional results directly related to the project) but also to do promotion among the capabilities of the project. Thereby the active participation (both, demo, sponsorship…) was the central measure to promote TNAs and raise attractiveness of VISIONAIR. By showcasing the interconnectivity of VISIONAIR partners, but also the involvement of external TNA participant, the project could be represented in an extraordinary way. The list of events were VISIONAIR attended is obviously fully reported in the periodic reports and is not reproduced here.
Task 2.3 Invitation & setting up for external proposal
This task is involved making sure the website was up to date with the call for proposals, the template form to submit proposal, regular proposal review meetings, request and collate reviewer feedback, and managing the completed TNAS.
Led by Cranfield, the review committee included representatives from Grenoble INP, HLR and Technion, Israel Institute of Technology. The “Proposal management/selection” area in the VISIONAIR site, managed by Grenoble, facilitated the operations of the committee. The targets for this task have been achieved.
The process of collecting the feedback from the host and the reviewers for a visit involved standard templates which were designed and used in this process to collate the information.
Task 2.4 Open forum development
This first open forum was held at Grenoble INP during the kick off in March 2011. The second open forum took place in Poznan in 2014. It was an extra day associated to the kickoff meeting or annual general assembly, fully open to any interested person when we mainly presented goals, and projects results. It has been two powerfull dissemination days.
Task 2.5 Organise an Associate Club
The Associate Club members are listed on the website and of four types: industrial, academic, visualisation vendor, scientific contributor. There were a number of ways to engage with the Associate Club members. For example, we shared different videos registered during the VISIONAIR seminars in order to enhance their knowledge about VISIONAIR activities and resources.
Members of the Club were also invited to participate to special event during conferences with specific VISIONAIR sessions.
They also have been invited to participate during the 1st General Assembly in Twente, the 2nd in Stuttgart, the 3rd in Budapest and the 4th in Rennes.
The 2nd to the last years has been devoted to the enlargement of the Club of Partners with the integration of EMIRAcle as a member of CVNET, the “Color and Vision Network”. On the flow of messages transmit in CVNet, EMIRAcle selected the interesting message for the VISIONAIR members, transmitted the messages to them and to the Club of Partners, and invited the expeditor of the message to reach the VISIONAIR Club of Partners.
As a network of Higher Education institution, EMIRAcle has the deal to promote VISIONAIR for their members who are not already partner in the VISIONAIR Infrastructure (13 EMIRAcle members on 25), and this is mainly outside Europe. The special links between EMIRAcle and the Asian countries permits the dissemination of the VISIONAIR infrastructure in Thailand and in India.
A first keynote has been given by Pr. Serge Tichkiewitch during the Conference for the 56th anniversary of the King Mongkut University of Technology North
Bangkok, the 19th of February 2014 (Figure 7) with title: “The European VISIONAIR Infrastructure”. Among the participants was the member of the Design, Manufacturing and Innovation (DMI) network, a network of 15 Thai Universities on the
governance of the National Science Technology and Innovation Policy Office (STI), Bangkok, Thailand, STI being one of the EMIRAcle members. The result of this conference was a proposition for a Program in Higher Education and Research SIAM 2015 between the Chiang Mai University (Thailand) and VISIONAIR Grenoble (France) on “Virtual Cultural Heritage - Digital Artifact Representation
in Collaborative Environment”, accepted in January 2015.
Using the capacity of the SCITE application to disseminate the research papers written by the members of the VISIONAIR Infrastructure and of its Partner’s Club, we add a special Web page about “Bibliography on Visualization” on the VISIONAIR web site, with the capacity to get the papers sort by Titles, Authors or Keywords. 77 papers concerning 186 keywords and written by 255 authors are available on the actual site.
List of Websites:
www.infra-visionair.eu