Periodic Reporting for period 2 - eTryOn (Virtual try-ons of garments enabling novel human fashion interactions)
Période du rapport: 2021-10-01 au 2022-12-31
The way people have been experiencing fashion has changed minimally over the past decades, mostly with the introduction of social media and in the direction of fashion marketing. The mission of eTryOn is to modernize the way people create, consume and experience fashion items (clothes) by offering novel Human-Fashion-Interaction (HFI) applications. In this direction, our objective has been to modify 3 existing experiences (creative, social, shopping) with novel HFI apps with the use of XR technologies. More specifically, three interactive solutions have been developed: i) the VR Designer app, enhancing the creative experience: a VR desktop application targeting fashion designers, with the aim to facilitate them throughout the creative process of garment design by offering immersive and realistic visualizations of the digital garments and their fitting on animated 3D avatars, ii) the Dress Me Up app, enhancing the social experience: a web application for social media users (e.g. influencers), allowing them to virtually change their outfit in an image by selecting from a pool of digital garments and then upload it to social media, and iii) the Magic Mirror app, enhancing the shopping experience: an AR mobile app enabling virtual tryons of garments during online shopping that aims to recreate at home the experience of buying clothes from a physical store. In developing these 3 apps, the main research, innovation and technological endeavours of eTryOn were to develop technology for: a) generating personal 3D avatars through a self-scanning app, b) automatically simulating the interaction between 3D user avatars and digital garments (i.e. size fitting and visualization of interactions during body movements), and c) extracting fashion insights from user preference data and generating fashion recommendations.
The outcome of the project in the form of Software is available as Open Source, and can be accessed through eTryOn's website relevant page:
https://etryon-h2020.eu/software/
The outcome of the project in the form of Software is available as Open Source, and can be accessed through eTryOn's website relevant page:
https://etryon-h2020.eu/software/
eTryOn developed 3 novel XR apps: i) the VR designer app, a desktop VR application targeting fashion designers, offering immersive and realistic visualizations of their digitally designed garments and how they fit on animated 3D avatars, ii) the Dress Me Up app, a web application for social media users (e.g. influencers), allowing them to virtually change their outfit on a photo by selecting from a pool of digital garments and then upload it to social media, and iii) the Magic Mirror app, a mobile-based AR magic mirror enabling virtual tryons of garments during online shopping that aims to recreate at home the experience of buying clothes from a physical store.
In creating these 3 apps and their components, we have developed a body self-scanning software with significant improvements in the accuracy of the creation of 3D body models, that can produce a dressable 3D avatar for use in the Dress Me Up application. Afterwards, relying on real garments from past, ongoing and new collections, we created digital assets that can be used in game engines, and are subsequently used in the VR Designer and Magic Mirror apps that run on the Unity game engine. Regarding the simulation of the developed avatars and the selected garments, the transformations of the aforementioned garments in forms compatible for simulation in game engines and AR frameworks were defined. For extracting fashion insights, we have developed a Machine Learning model for the detection of categories and attributes within fashion imagery. Furthermore, the aforementioned model in combination with past trend data were used to form a fashion trend analysis and prediction model. On combining all the aforementioned technologies in standalone XR apps, we have set up a cloud-native, serverless-first architecture design delivered and iteratively kept up-to-date as the project evolved and incorporated an Architectural Decision Records methodology. Furthermore, detailed technical diagrams have been delivered with clear distinctions of the cloud functions that were implemented that aided all three XR apps. Regarding the development of the applications, we firstly analysed the options and selected the platforms for the software implementation of each app. All the aforementioned technical work relied on the detailed user requirements that were gathered through questionnaires directed to the target audience of each app. Cloud function integration, API implementations and the development of the three apps has been finalized. Furthermore we defined a clear plan of testing the applications in our pilot sessions and carried out several dissemination activities on various social media channels to promote said sessions. We carried out the three different pilots and we got feedback from our end users, in the form of questionnaires (subjective) as well as from analytics services that we have set up (objective).
In creating these 3 apps and their components, we have developed a body self-scanning software with significant improvements in the accuracy of the creation of 3D body models, that can produce a dressable 3D avatar for use in the Dress Me Up application. Afterwards, relying on real garments from past, ongoing and new collections, we created digital assets that can be used in game engines, and are subsequently used in the VR Designer and Magic Mirror apps that run on the Unity game engine. Regarding the simulation of the developed avatars and the selected garments, the transformations of the aforementioned garments in forms compatible for simulation in game engines and AR frameworks were defined. For extracting fashion insights, we have developed a Machine Learning model for the detection of categories and attributes within fashion imagery. Furthermore, the aforementioned model in combination with past trend data were used to form a fashion trend analysis and prediction model. On combining all the aforementioned technologies in standalone XR apps, we have set up a cloud-native, serverless-first architecture design delivered and iteratively kept up-to-date as the project evolved and incorporated an Architectural Decision Records methodology. Furthermore, detailed technical diagrams have been delivered with clear distinctions of the cloud functions that were implemented that aided all three XR apps. Regarding the development of the applications, we firstly analysed the options and selected the platforms for the software implementation of each app. All the aforementioned technical work relied on the detailed user requirements that were gathered through questionnaires directed to the target audience of each app. Cloud function integration, API implementations and the development of the three apps has been finalized. Furthermore we defined a clear plan of testing the applications in our pilot sessions and carried out several dissemination activities on various social media channels to promote said sessions. We carried out the three different pilots and we got feedback from our end users, in the form of questionnaires (subjective) as well as from analytics services that we have set up (objective).
In eTryOn, we have brought together two significant complementary but yet disparate industries; fashion and gaming. By working closely with the fashion brands, we have developed technologies and know-how on utilizing garments designed for production in game engine environments, and all this without disrupting the designers’ workflow. In this direction, we have developed methodologies in order to transfer the accurate garment simulations from the design software into game-engine ready formats. In this way, we bring the best of the two worlds together, benefiting from the accuracy of the design software and the interaction capabilities of game engines. This is also accompanied by innovations on 3D body scanning, by making available people mobile apps that can generate their personal 3D avatar through the use of a mainstream device, as well as novel fashion insights providing the designers with information on how popular a garment they design is expected to be.
As a result of eTryOn, the use of Interactive Technologies in fashion industry will be increased, further enhancing the number of European SMEs and start-ups who benefit from technology transfer and market opportunities. New opportunities for fashion marketing will be created, re-inventing the e-commerce experience for consumers, by providing customers products that are better suited for them and also providing solutions to significant fashion industry/market problems such as returns of online orders and trend detection for scheduling production, helping brands reduce apparel returns and increase sales conversion.
Moreover, through the VR designer app and eTryOn in general, we expect to convince more fashion brands turning to 3D design, which reduces significantly the cost garment design and reduces fabric waste. Furthermore, the development of the DressMeUp will facilitate the mainstream use of digital fashion (digital collections for example), joining the new era of digital fashion in metaverse and gaming industry. It will also allow for reducing material waste since fashion lovers need not to buy anymore an outfit for a one-time-show. The Magic Mirror app will provide innovative solutions in e-commerce allowing for more fitted purchases and hence less returns. eTryOn has therefore begun to pave the way for potentially ground-breaking approaches, contributing to a more sustainable model in the fashion industry.
As a result of eTryOn, the use of Interactive Technologies in fashion industry will be increased, further enhancing the number of European SMEs and start-ups who benefit from technology transfer and market opportunities. New opportunities for fashion marketing will be created, re-inventing the e-commerce experience for consumers, by providing customers products that are better suited for them and also providing solutions to significant fashion industry/market problems such as returns of online orders and trend detection for scheduling production, helping brands reduce apparel returns and increase sales conversion.
Moreover, through the VR designer app and eTryOn in general, we expect to convince more fashion brands turning to 3D design, which reduces significantly the cost garment design and reduces fabric waste. Furthermore, the development of the DressMeUp will facilitate the mainstream use of digital fashion (digital collections for example), joining the new era of digital fashion in metaverse and gaming industry. It will also allow for reducing material waste since fashion lovers need not to buy anymore an outfit for a one-time-show. The Magic Mirror app will provide innovative solutions in e-commerce allowing for more fitted purchases and hence less returns. eTryOn has therefore begun to pave the way for potentially ground-breaking approaches, contributing to a more sustainable model in the fashion industry.