Project description
Automatically created 3D content for various applications
Recent advancements in image generation methods, such as generative adversarial networks and diffusion models, have greatly improved realism. However, 3D modelling for computer graphics and immersive experiences remains behind. The ERC-funded Gen3D project aims to bridge this gap by automatically generating 3D content for virtual worlds, enabling flexible rendering from different viewpoints while maintaining real-world visual fidelity. The project will support various applications, including video games, movies, augmented reality (AR), virtual reality (VR), computer-aided design (CAD), architectural visualisation, and medical imaging. It will focus on developing 3D generative models capable of producing polygon meshes with textures and material properties while enhancing control and editability through conditional generation for both novice and expert users.
Objective
In recent years, we have seen a revolution of learning methods that generate highly-realistic images, such as generative adversarial neural networks, autoregressive methods, or diffusion models (e.g. DALL-E, Stable Diffusion, Runway, etc). Unfortunately, the vast majority of these methods are tailored towards the 2D image domain, while their respective 3D counterparts 3D models that fuel computer graphics applications, and enable visually immersive experiences remain in their infancy.
In this proposal, we tackle the challenge of automatic generation of 3D content for virtual worlds. Such 3D generated content enables versatility, with flexible rendering from arbitrary viewpoints that match the visual fidelity of the real world. We focus on 3D content creation for visually immersive experiences for a much wider audience in myriad applications, such as video games, movies, AR/VR scenarios, CAD modeling, architectural & industrial design, and medical applications. We believe that the key towards automated, high-fidelity content creation lies in developing new machine learning techniques to transform 3D content generation.
(A) We will develop 3D Generative Models that output 3D polygon meshes, along with their surface textures and material properties, highlighting generation of 3D content that can be directly consumed by modern graphics pipelines.
(B) To train our 3D generative models to reflect the complexity and diversity of real data, we will devise methods for Supervision from Images and Videos. The key challenge here is that such collections of images and videos are by nature incomplete projections of the underlying 3D world, thus requiring learning paradigms that generalize across partial instances.
(C) We will research techniques that provide Control and Editability through Conditional Generation. In particular, we will focus on conditional input from both novice (e.g. text-based editing) and expert (e.g. based on existing authoring tools) users alike.
Fields of science (EuroSciVoc)
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
CORDIS classifies projects with EuroSciVoc, a multilingual taxonomy of fields of science, through a semi-automatic process based on NLP techniques. See: The European Science Vocabulary.
- social sciences political sciences political transitions revolutions
- engineering and technology environmental engineering energy and fuels
You need to log in or register to use this function
Programme(s)
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
Multi-annual funding programmes that define the EU’s priorities for research and innovation.
-
HORIZON.1.1 - European Research Council (ERC)
MAIN PROGRAMME
See all projects funded under this programme
Topic(s)
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Calls for proposals are divided into topics. A topic defines a specific subject or area for which applicants can submit proposals. The description of a topic comprises its specific scope and the expected impact of the funded project.
Funding Scheme
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
Funding scheme (or “Type of Action”) inside a programme with common features. It specifies: the scope of what is funded; the reimbursement rate; specific evaluation criteria to qualify for funding; and the use of simplified forms of costs like lump sums.
HORIZON-ERC - HORIZON ERC Grants
See all projects funded under this funding scheme
Call for proposal
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
Procedure for inviting applicants to submit project proposals, with the aim of receiving EU funding.
(opens in new window) ERC-2024-COG
See all projects funded under this callHost institution
Net EU financial contribution. The sum of money that the participant receives, deducted by the EU contribution to its linked third party. It considers the distribution of the EU financial contribution between direct beneficiaries of the project and other types of participants, like third-party participants.
80333 Muenchen
Germany
The total costs incurred by this organisation to participate in the project, including direct and indirect costs. This amount is a subset of the overall project budget.