Skip to main content
European Commission logo print header

Machine Understanding for interactive StorytElling

Project description

Challenging current Thinking

The MUSE project will introduce a new way of exploring and understanding information by "bringing text to life" through 3D interactive storytelling. Taking as input natural language text like children's stories or medical patient education materials, MUSE will process the natural language, translate it into formal knowledge that represents the actions, actors, plots and surrounding world, and then render these as virtual 3D worlds in which the user can explore the text through interaction, re-enactment and guided game play.To enable such a system, MUSE will make targeted advances in natural language processing that enable the translation of natural language text to the necessary knowledge representations, as well as targeted advances in the action representation and story planning necessary for interactive storytelling. In natural language processing, MUSE will develop new techniques for finding explicit action structures in text and combining them with implicit action structures inferred from the context based on probabilistic models of translation and automatic methods for acquiring world knowledge from large corpora. In interactive storytelling, MUSE will develop action and object representations that bridge the gap between natural language and virtual worlds, and will create advanced techniques for planning virtual world stories given inconsistent and incomplete information.The proposed methodology will be evaluated and showcased on two scenarios: one for creating immersive children's stories from text and one for allowing medical patients to interact with patient education materials. Comparable to the invention of symbolic writing systems several millennia ago, MUSE contributes to a novel symbolic system communicating natural language utterances.

Call for proposal

See other projects for this call


Oude markt 13
3000 Leuven

See on map

Vlaams Gewest Prov. Vlaams-Brabant Arr. Leuven
Activity type
Higher or Secondary Education Establishments
Administrative Contact
Tine Heylen (Ms.)
EU contribution
No data

Participants (4)