Periodic Reporting for period 1 - DLift (Lifting Design Drawings to 3D)
Periodo di rendicontazione: 2024-04-01 al 2025-09-30
Drawing – or sketching – is a fundamental tool of product design. By tracing a few pen strokes, designers can quickly explore 3D shape ideas, and communicate these ideas to colleagues and clients. However, while 2D drawings are easily understood by humans, they are currently not interpretable by computers. To confront their ideas with physical reality, designers have to separately create 3D models that form the necessary input for engineering tools, such as numerical simulation and 3D printing. But while drawing is visceral and quick, 3D modeling requires careful planning, precision and constraints to manipulate 3D shapes on screen. Skilled 3D modelers often need several hours to convert a drawing into a 3D model, at an hourly rate of 50euros on average, making 3D modeling a major bottleneck that hinders rapid iterations between ideation and prototyping. As a consequence, even though the fluidity of drawing stimulates the creativity of designers, only a small selection of the ideas generated during design exploration get evaluated during design engineering.
The ERC Starting Grant D3 (https://ns.inria.fr/d3/index.html(si apre in una nuova finestra)) succeeded at bridging design exploration and design engineering by offering the first algorithm capable of automatically lifting 2D design drawings to 3D. This technology brings unique benefits to product design:
- Accelerating design. During design exploration and presentation, our technology allows designers to visualize and refine their drawing from any viewpoint. In contrast, in the current workflow, designers need to re-draw the same shape multiple times to show how it would look in different contexts.
- Facilitating collaboration between designers and 3D modelers. During transition to 3D modeling, our technology allows designers to communicate their intent to 3D modelers more effectively, supporting faster iterations between team members.
- Accelerating 3D modeling. During 3D modeling, our technology provides an initial reference of the shape and its proportions, which 3D modelers can build upon instead of using 2D drawings as mere visual guidance.
The objective of DLift is to demonstrate how our technology for reconstructing drawings in 3D can streamline the Computer-Aided-Design (CAD) workflow, and to take the first steps in industrializing this technology. DLift will fundamentally transform the design workflow by freeing designers from the burden of mastering 3D modeling, and by freeing 3D modelers from the burden of deciphering design drawings. By providing a seamless transition from drawings to 3D models, DLift has the potential to drastically increase the creativity and productivity of design teams, empowering them to explore a larger design space during the initial phases of product design.
The ERC Starting Grant D3 (https://ns.inria.fr/d3/index.html(si apre in una nuova finestra)) succeeded at bridging design exploration and design engineering by offering the first algorithm capable of automatically lifting 2D design drawings to 3D. This technology brings unique benefits to product design:
- Accelerating design. During design exploration and presentation, our technology allows designers to visualize and refine their drawing from any viewpoint. In contrast, in the current workflow, designers need to re-draw the same shape multiple times to show how it would look in different contexts.
- Facilitating collaboration between designers and 3D modelers. During transition to 3D modeling, our technology allows designers to communicate their intent to 3D modelers more effectively, supporting faster iterations between team members.
- Accelerating 3D modeling. During 3D modeling, our technology provides an initial reference of the shape and its proportions, which 3D modelers can build upon instead of using 2D drawings as mere visual guidance.
The objective of DLift is to demonstrate how our technology for reconstructing drawings in 3D can streamline the Computer-Aided-Design (CAD) workflow, and to take the first steps in industrializing this technology. DLift will fundamentally transform the design workflow by freeing designers from the burden of mastering 3D modeling, and by freeing 3D modelers from the burden of deciphering design drawings. By providing a seamless transition from drawings to 3D models, DLift has the potential to drastically increase the creativity and productivity of design teams, empowering them to explore a larger design space during the initial phases of product design.
The goal of DLift is to demonstrate the applicability of our 3D reconstruction algorithm within the established 3D modeling workflow, and to take the first steps towards licensing this technology to CAD software editors.
To achieve this goal without being tied to a specific software editor, our strategy has been to implement a demonstrator as an add-on for the 3D modeling software Blender. Blender is a well-documented, industry-grade, open-source CAD software. It offers both state-of-the-art 3D modeling capabilities as well as easy integration of custom functionality. In addition, Blender includes a rich drawing interface – called Grease Pencil. We have leveraged this interface to allow users draw shapes in perspective, and used our algorithm to lift the drawing to 3D for subsequent modeling using Blender tools.
We have used our demonstrator to create a gallery of 3D drawings, and showcased its capabilities to industrial contacts. We have also compared our reconstruction algorithm to recent AI-based models, which revealed complementary strengths between their data-driven priors and our geometric optimization.
We released the source code of our demonstrator under a research-only license, along with examples of 3D drawings created with it:
https://gitlab.inria.fr/D3/blender-addon-symmetry-sketch(si apre in una nuova finestra)
To achieve this goal without being tied to a specific software editor, our strategy has been to implement a demonstrator as an add-on for the 3D modeling software Blender. Blender is a well-documented, industry-grade, open-source CAD software. It offers both state-of-the-art 3D modeling capabilities as well as easy integration of custom functionality. In addition, Blender includes a rich drawing interface – called Grease Pencil. We have leveraged this interface to allow users draw shapes in perspective, and used our algorithm to lift the drawing to 3D for subsequent modeling using Blender tools.
We have used our demonstrator to create a gallery of 3D drawings, and showcased its capabilities to industrial contacts. We have also compared our reconstruction algorithm to recent AI-based models, which revealed complementary strengths between their data-driven priors and our geometric optimization.
We released the source code of our demonstrator under a research-only license, along with examples of 3D drawings created with it:
https://gitlab.inria.fr/D3/blender-addon-symmetry-sketch(si apre in una nuova finestra)
Product design involves multiple phases, where different experts use different tools to progressively refine a visual description of a new product. During the initial ideation phase, designers make extensive use of freehand drawings to describe their ideas. Once a concept is sufficiently mature, 3D modelers use Computer-Aided-Design (CAD) tools to convert the design drawings into 3D models suitable for simulation and manufacturing. But a single design drawing only shows the shape from a single viewpoint! As such, drawings only serve as visual inspiration for 3D modelers who need to interpret how the shape would look like from other vantage points. For this reason, designers and 3D modelers often need multiple iterations to understand each other, designers providing additional drawings to clarify their intent, and 3D modelers updating their models to better match the provided drawings.
Our technology provides a unique answer to the above challenges. On the one hand, it maintains the fluidity of freehand drawing, allowing designers to focus on their creative task rather than on system interactions. On the other hand, it allows design drawings to to be observed from any viewpoint, providing an in-situ reference for precise 3D modeling. Our demonstrator, implemented in Blender, shows how drawing reconstruction can be integrated into existing CAD software to bridge design ideation and 3D modeling.
Our technology provides a unique answer to the above challenges. On the one hand, it maintains the fluidity of freehand drawing, allowing designers to focus on their creative task rather than on system interactions. On the other hand, it allows design drawings to to be observed from any viewpoint, providing an in-situ reference for precise 3D modeling. Our demonstrator, implemented in Blender, shows how drawing reconstruction can be integrated into existing CAD software to bridge design ideation and 3D modeling.