Augmented Commerce, a game-changing augmented reality-based sales experience to disrupt the e-commerce industry

Project Information

Augmented Commerce

Grant agreement ID: 762347

DOI

10.3030/762347

Project closed

EC signature date 26 January 2017

Start date 1 February 2017

End date 31 July 2017

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost

€ 71 429,00

EU contribution

€ 50 000,00

Coordinated by

AUGMENT
France

Periodic Reporting for period 1 - Augmented Commerce (Augmented Commerce, a game-changing augmented reality-based sales experience to disrupt the e-commerce industry)

Reporting period: 2017-02-01 to 2017-07-31

Summary of the context and overall objectives of the project

Augment provides an augmented reality platform that connects retailers and manufacturers. On the retailer side, Augment integrates into their mobile applications and website to allow customers to visualize the products available in their own environment. On the manufacturer side, Augment retrieve the existing 3d content, clean it and optimize it to make it available to the retailers. If the content is not available, Augment will create it. Augment is already integrated into the applications of Fnac, Cdiscount, Leroy-Merlin and more.

The biggest challenge consists in scaling the database of products quickly enough, in an economically viable way. The existing process to produce 3d models cost in average 50€ per models. To build a full catalog with hundreds of thousands of products can quickly cost several millions of euros. In some categories, the turnover of products is higher than 40%, generating an on-going cost to maintain a decent coverage.

Solving that issue is key to bring augmented reality to a level where the consumer can check out any product he wants to buy. Augment tackled that challenge with our Deep Modeling Generator project.

The project revolves around the automatic generation of 3d models from the information available on an e-commerce website product listing. With pictures and meta-data we generate a matching 3d model that is then uploaded to our product database and linked to the real object. Once the object becomes available in 3d, it's distributed to the product pages of the retailers in which our technology is integrated.

We developped a working prototype of the deep modeler to validate the mechanism and the output. That step was completed in September 2016 with 3d models of doors, gates, television, and carpet created automatically from selected e-commerce websites. Hundreds of generated models were published on Augment compatible e-commerce mobile applications. Customer feedbacks was positive and validated the approach.

At the same time, our product distribution platform was gathering information about more than 2.5 million products that were prepared for our Deep Modeling Generator. With the database now ready, the resulting models validated by customers and Augment technology deployed into more than 1 million installed application, we can start the next phase of the project which consists in scaling the number of product families automated and improving the full workflow.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

Augment is building the leading platform for product visualisation and distribution across retail and e-commerce. We provide all the tools needed to create, manage and integrate Augmented Reality visualisation of products inside retailer’s own website and mobile app. We started phase I with two customers’ integrations, inside Cdiscount and Leroy-Merlin mobile applications. The first goal of Phase I turned out to test the market reaction to our offer and customers’ willingness to invest into 3D content.

We met with more than 100 retailers during that period, gathering feedbacks about usage of Augmented Reality in e-commerce, product content available and coverage expectation for their existing catalog. On those 100, already a dozen had experimented with Augmented Reality implementation. None of them deployed those experimentation in production, the main reason being the lack of process to scale 3D content. They mostly produced a few models with 3D designers, or tried to get the models from the manufacturers. They failed to find an economic model that made sense with both approaches.

We then analyzed the product catalogues and found the most number of products compatible with a deep modeling approach in furniture, home decor and Do-it-yourself retailers. We expect that approach to cover up to 20% of those kind of catalogs with its most simple approach.

Next step was to acquire more data on products from product content providers and the retailers themselves. We were able to get 5 more retailer integrations and did a partnership with GFK to get access to their content. We’ve gathered nearly 4 million products datasheet during that period.

At the same time we worked on a market study to analyze long term impact of our product and how it could be integrated in our marketing and sales pipeline. At the same we ran an IP analysis, that led Augment to write a patent describing our new method to build automatically 3D models out of product pages. The patent has been filled at the European level and is currently pending.

On the technical side, we tested different ways to create big volume of 3D models. We experimented with human 3D designers, creating 3D for CAD files sent by manufacturers; 3D designers working out of photographies of the products, usually available directly on the e-commerce website; photogrammetry which is the reconstruction of a 3D model out of usually more than 30 pictures of the product; and finally deep modeling. The result of this study is that the only economically viable approach to build a large database of 3D models from product page is to use automatic 3D modelling. With the turnover of products, the amount of models to produce just to keep the database up to date is not sustainable by a single retailer.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Augmented and Virtual Reality’s popularity is growing a lot. The set of medias the mass market is interacting with on a daily basis will require more and more content. This content is availability of 3D models. At the moment (2017), all the ways to create 3D models are expensive and don’t fit a vision in which we find a digital double for every single object in the real world.

During the Deep Modeling project, we expect to push the boundaries of content creation by scaling our approach to any family of products, and then to any object in general. If an artist can build a 3D model out of a few pictures, our Deep Modeler will be able to do the same. Once the mechanism is in place, it will be able to crawl the web and create a virtual duplicate to every object that appears on a photo.

Our vision is that Mixed Reality where digital and real people are sharing a virtual or real location will be spread a lot. To interact properly in such a setting, we need to provide an accurate representation of the real world, including the objects present in the location. Once this condition is realized, a whole range of new applications will emerge and will bring tele-commuting to jobs and usages that couldn’t do it yet. For example, an interior designer will be able to meet his customer virtually in the living room that needs remodelling. The customer is physically present in the room, and he sees the virtual avatar of the interior designer. They can both communicate and the interior designer can suggest modifications in the room that will be immediately visible by the customer. All a sudden you can be a home decorator, working from your own home and visiting multiple clients a day.

Such an approach allows the interior designer to live far from his target customer, visiting more customers per day. Which mechanically will drive down the price of their services and open it to more households. The same reasoning can be applied to all sort of jobs that currently can be done only in person.

The other impact of the project is to allow a customer buying from an e-commerce website, to be sure about the product he wants to buy. Since he saw the product exactly like it’s supposed to be, their is fewer reasons to return it. This will reduce the return cost for the retailer and the amount of travel the product will do for nothing.

augmented-commerce.jpg