The project started in January 2017, setting up the collaboration with the Visual Lab of the Robotics Institute at the Instituto Técnico Superior in Lisbon (IST).
The MoU was signed Sept. 2017. In Sept. 2018 the ISTb ecame second beneficiary.
Subproject 1: human psychophysics.
My multisensory two-axis vestibular chair became available for the psychophysics June 2018. In Sept 2017, PhD 1, and per Oct. 2018, Postdoc 1, were appointed. First results on neural mechanisms of sound-localisation in noisy environments, and Bayesian mechanisms for audiovisual integration were published in 2017-2019. The PI held several presentations on this work in international conferences and invited seminars, e.g. at the NCM meetings in Santa Fe, USA, in Toyama, Japan, in Rovereto, Italy, in Kosice, Slovakia, and in Alicante, Spain.
We hired prof A Snik per Sept 1, 2017 as an expert audiologist, to work on sensory-deprived patients in collaboration with our applicants. As world-recognized expert on auditory technology and audiology, he fits perfectly in the Action’s aims. In Jan 2019, we attracted postdoc 2 to work on audio-visual psychophysics and plasticity/adaptation.
To set up the auditory patient work, we hired two PhD researchers from Jan-June, 2019 to perform sound-localization studies with hearing-impaired patients. Postdocs 1 and 2also performed auditory motion experiments.
Subproject 2: Computational modelling
During the first months of the Action (Feb-May, 2017), the PI appointed a PhD student to work on a computational model of the midbrain by implementing a novel spiking neural network algorithm. Six manuscripts have arisen from this work. In April 2019, the PI appointed Postdoc 3 who has extended the spiking network modelling to 3D eye-head coordination.
Subproject 3: Humanoid robotic model.
The collaboration with prof Bernardino went very well. Between April–Oct 2017 a master student designed and tested a first prototype robotic eye (Thesis report, see Website). From April 2018 to Oct. 2019, this work continued with two new students: one working on kinematics and mechanical improvements; the other on visual-image processing and positional stabilisation. The coordinator and Bernardino recruited PhD2 for Sub-project 3 in Sept. 2018 and we actively searched for Postdoc 4, who could start by the end of 2020.
During the final period 4 (months 54-72) we worked towards a successful end of our project, despite the strong detrimental influence of the Covid-19 epidemic, which had seriously impaired the experimental work, both in Nijmegen and in Lisbon. At the time of writing (April 23, 2023), 47 research papers resulted from this action, with 4 additional papers currently under review, which is an excellent result.
In addition, the PI took the initiative to publish (together with 7 co-authors from the EU and USA) the seminal modelling work of prof David A Robinson (this world-leading researcher passed away in Nov. 2018 at the age of 92) on the Oculomotor System, as a full issue of Elsevier’s Progress in Brain Research (Vol. 267, 435 pages). It appeared Feb 2022.
The work on the robotic eye-head system in Subproject 3 in Lisbon went very well and is still ongoing: PhD-2 constructed a highly improved biomimetic prototype of the human eye with six muscle and motors, and seven master students finished their research theses with valuable results (see project’s website for all thesis reports). Four papers have been submitted in 2022/2023 on our robotic eye, and two more are currently in preparation. The PI aims to continue this fruitful collaboration in his new ERC project proposal.
The project has an open website:
http://www.mbfys.ru.nl/~johnvo/OrientWeb/orient.html(odnośnik otworzy się w nowym oknie) with the project’s background, ongoing work, demo’s, prototypes, results, the team, research papers and student progress reports are updated regularly.
