Final Report Summary - DURAFILE (Innovative Digital Preservation using Social Search in Agent Environments)
During our lives, we generate a lot of personal information such as photos, music, text documents and videos which link us with our past. This data that used to be tangible is now digital information stored in our computers, which implies a software dependence to make them accessible in the future. Technology, however, constantly evolves and goes through regular shifts, quickly rendering various file formats obsolete. The need for accessing data in the future affects not only home users but also organisations. In Europe, the legislation requires that organizations retain their digital information for periods ranging from few months to several years. These organizations must ensure access to all data for several reasons: audits, access to past legal documents or scientific data, records of different events, etc. Digital preservation (DP) represents the management of digital information over time to guarantee their accessibility and preserved quality. It includes processes and activities that ensure access to information and all types of records, scientific and cultural heritage that is stored in digital formats. Digital preservation offers guarantee the accessibility of our personal documents in the future. In a digital environment, a reliable preservation plan and the ability to adapt to fast changing technology are essential for maintaining data collections in the long term.
The DURAFILE project aims to offer a novel software solution preserving both personal digital information and the digital information generated by small and medium organizations. As a standalone application installed on personal computers, DURAFILE will regularly scan all files – working in the background – similarly to anti-virus programs detecting obsolete files or files at risk of obsolescence. The software additionally, will provide specific solutions with high security requirements for those organizations that need to preserve automatically the integrity and accessibility of large, catalogued and searchable archives. As an easy to use tool, DURAFILE will facilitate the preservation of valuable information, and will provide a long term access to data for organizations and individual users.
Project Context and Objectives:
The Project was divided into two periods: the first period from M1 to M9 and the second period from M10 to M24.
The work performed during the first period was concentrated in describing the framework for the DURAFILE platform. The initial effort was dedicated to the definition of the user and the system requirements (including security issues on the private data), identifying different characteristics of the possible B2B and B2C scenarios and implementing a new Social Search algorithm adapted for Digital Preservation. The work of the first 9 months of the first period of the DURAFILE project included the following tasks was:
• Study of the state of the art in Digital Preservation (WP1). The work included the identification of the state of the art, models, methods, methodologies and applications of agents in digital preservation and the identification, adaption and development of models and methods needed to define DURAFILE agents developed in this project. The study also included an outline the final market for the digital preservation, identifying the different characteristics of the end consumers in the B2B and B2C scenarios.
• Definition of the user’s requirements for the DURAFILE platform (WP1). The work included the collection of the users and the SMEs requirements and the definition of a platform architecture with a commercial approach. Algorithms for monitoring digital objects were defined. In the new DURAFILE platform design, the monitoring of digital objects is defined in the preservation process done at the DURAFILE Server. A specific ontology for digital preservation was also defined. The ontology generated comprises a taxonomy that classifies the knowledge and all the related concepts and types of relationships that enriches each topic.
• Development of a Security/trust policy working with preservation plans (WP2). The work included the selection of the protocols for securing the data transmission, the data storage and the execution of Digital Preservation Plans and the creation of a Trust model used for the Experts in digital preservation.
• Development of Social Search algorithm for digital preservation (WP3). During the period, a Social Search Algorithm for Digital Preservation purposes was implemented. The work included a library of semantic algorithms which will be used for the implementation of the Harvesting Agent (scheduled in Task 4.4) and the communication mechanism between the agents of the Experts Community.
On the other hand, the work performed during the second period of the project (from 10 to M24) was concentrated in the implementation of the DURAFILE platform based on the requirements and the framework studied during the first period. During this period, dissemination activities were taken by the SMEs in order to promote the platform among possible customers. An Exploitation Plan was discussed among the owners in order to fix the rules for a possible commercialization. Next tasks provided a more detailed explanation of the work performed during the second period:
• Development of DURAFILE platform (WP4). During the second part of the project, UTA was involved in the development of the Recipe Server which contains all the infrastructure of the DPPs executions. UDG which has more experience in software programming assisted UTA during the last part of the implementation. UDG was mainly involved in the software platform which includes the DURAFILE Server and the needed infrastructure for connecting it with the Recipe Server. ATEKNEA participated in the implementation and integration of the Harvesting Agent which is the responsible for facilitating file extensions information to the DURAFILE users.
• Development of a GUI for the DURAFILE platform (WP5). ATEKNEA with the assistance of UTA and UDG created the interface for the Web and Desktop DURAFILE application. The SMEs of the consortium provided their feedback and assisted ATEKNEA modifying the GUI in order to have a user friendly interface. ATEKNEA coordinated the translations of the interface menus into the languages of the members of the consortium (Greek, Estonian, Turkish, Spanish, Catalan, French and Romanian).
• Test and validation of the DURAFILE platform. Validation tests were performed by the SMEs after the training sessions. From the feedback provided by the SMEs, some errors were solved and some functionalities were modified according the SMEs.
• Training activities. A training plan to ensure that the SMEs assimilate the project result including the technical, socio-economic and commercial benefits of the DURAFILE technology were created. All members of the consortium were actively involved under the coordination of ASSIST and assisted by ATEKNEA and the representatives from UDG and UTA.
• Dissemination activities. The project website (www.durafile.eu) with updated information about the project and related news, was created and served as vehicle of communication and dissemination of the project and its results. All consortium members have been actively involved in dissemination activities, like articles in magazines, newsletters to different private and public entities, and participation in conferences and workshops mainly done by ATEKNEA, UDG and UTA.
• An Exploitation Plan. The SMEs were actively involved, under the coordination of EASY as exploitation leader, in the creation of an Exploitation Plan for the DURAFILE software setting the rules for a future exploitation.
The main objective of this Project was the development of a software platform for home users and small/medium companies for easily maintain their multimedia files. The platform includes expert users, which are users than create and interchange Digital Preservation Plans among them in order to find suitable preservation plans for obsolete or with risk of obsolescence multimedia files. The software assures that the users’ information remains available in the future, no matter how the technology evolves or current data formats become obsolete. DURAFILE is available as a web application that scans the files of the home users or small/medium companies but also as a downloable client for the most common operating systems: Windows, Linux and MACOS. The technology was validated by the SMEs of the consortium for text, audio, video and image multimedia files.
In order to achieve this overall goal, the project achieved different technical some operational objectives within the 24-month-time-frame. The list of results is presented per WP:
• WP1.- Definition and specification of the DURAFILE platform. The objective of this WP was the study of the state of the art in Digital Preservation, the possible B2B and B2C relations and the collection of SMEs and end user requirements. This collected data was used for creating the specification of the DURAFILE platform. The specific results of this WP included:
o A Study of the use of Intelligent Agents for Digital Preservation.
o The Identification possible B2B and B2C scenarios.
o The DURAFILE Platform specifications.
• WP2.- Trust & Security. The objective of this WP was the study of the different security methods for assuring that the stored data is only accessed by the owner. The consortium studied the security when the data is transmitted from the user computer to the DURAFILE server and the security when the data is stored in the server by using encryption. Security executing Digital Preservation Plans were studied, including some data for assuring the Digital Preservation Plan does not contain any malicious code and does what it is supposed to do. A trust and reputation model used by the experts was defined. The specific results of this WP included:
o A study of the different data security issues needed for assuring the privacy of the data stored in the DURAFILE Server. That included security in data transmission, in data storage and in executing Digital Preservation Plans.
o The creation of a trust model for using it in the Expert Network.
• WP3.- Semantic algorithms & Social Search. The objective of this WP was the creation of a library of algorithms for monitoring certain web pages or forums (and that will be used in WP4 by the harvesting agent), the creation of a mechanism for communicating the agents and the development of a Social Search algorithm for Digital Preservation. The specific results of this WP included:
o The design of the Semantic Search Algorithms for helping in the creation of new Digital Preservation Plans.
o The design of the different mechanisms for the communication between the agents of the Expert Network.
o The design and implementation of the Social Search algorithms.
• WP4.- DURAFILE Platform. The objective of this WP was the development of the DURAFILE software for home users and small/medium companies, based on the specifications discussed during the first period of the project. The WP included the implementation of the Harvesting Agent in order to find information about files format. The specific results of this WP included:
o A study of the functional requirements of the platform when automate the execution of Digital Preservation Plans.
o The implementation of the DURAFILE platform based on the requirements obtained during the first period of the project (WP1 & WP2).
o The Implementation of the Harvesting Agent, based on the requirements of WP3.
• WP5.- Desktop and Web GUI design. As a result of this WP a user-friendly interface was created in order to facilitate to these non-technical users the use of a Digital Preservation tool. 2 versions of the DURAFILE platform were released:
o DURAFILE Desktop application. This is the desktop application version that works similar to Dropbox. There are Windows, Linux and Mac installer that can be easy installed in a home or company computer. Once installed the DURAFILE software works on the background checking the obsolete files and showing the notifications in the main bar without disturbing the user.
o DURAFILE Web application. Similar to Dropbox that also has a web application, DURAFILE users can use a web application where user can access to their synchronized files through the Web interface.
The specific results of this WP included:
o The DURAFILE Desktop application.
o The DURAFILE Web application.
o DURAFILE translated to the languages of the consortium: French, Romanian, Turkish, Estonian, Greek, Catalan and Spanish.
o Windows, Linux and MAC executable versions.
• WP6.- Field test & Platform validation. The result of this WP was the validation of the DURAFILE platform in order to establish the difficulty of using this type of software for users which are not familiar with the Digital Preservation processes. The validation tests done at the end of the project were used for improving the quality of the software. All partners participated in this validation. The specific results of this WP included:
o A validation of the DURAFILE Desktop application by using image, audio and text files.
o A validation of the DURAFILE Web application by using image, audio and text files.
o The creation of a validation plan.
o DURAFILE GUI improvement based on the validation feedback.
o DURAFILE server updates based on the validation feedback.
• WP7.- Facilitating the up-take of results. The objective of this WP was the train of the SMEs, the dissemination of the results at EU level and definition the ways to protect and exploit foreground. A plan for Knowledge transfer to SMEs was defined. Several dissemination materials were developed (logo, video, press notes, etc.) and used in actions performed at EU level. A Plan for the Use and Dissemination of Foreground was prepared, collecting the individual intentions of each partner and describing actual achievements in dissemination and exploitation of results. All partners participated in the training and dissemination. The specific results of this WP included:
o A Training plan for assuring that the knowledge is transferred from the RTDs to the SMEs.
o Training material: CD, Technical presentation, 3 videos (a general video of the project and 2 videos more commercial oriented for home users and small/medium companies.
o A DURAFILE user guide.
o A DURAFILE Brochure.
o 2 Training sessions from the RTDs to the SMEs.
o An agreement among the SMEs for further exploitation of the DURAFILE software.
o Several press releases disseminating the project (including TV and radio).
o The creation of a project website with updated information about the project and Digital Preservation news.
o An interim PDUF
The only estimation of the market size for digital preservation has been done at global level. According to the Open Planets Foundation, the global demand for digital preservation in the global market is over $1 billion. The digital preservation market also includes digitizing (converting conventional to digital format) and hardware obsolesce. However, the type of digital preservation targeted by DURAFILE – digital preservation of files – is likely to account for at least one third of this demand. Therefore, if Europe is the first to offer an easy-to-use intelligent file conversion programme, European companies can reap substantial benefits as the first to enter the market and thereby gaining a competitive advantage.
The DURAFILE software offers a novel solution for preserving both personal digital information and the digital information generated by small and medium organizations. As a standalone application installed on personal computers, DURAFILE regularly scans all files – working in the background – similarly to anti-virus programs detecting obsolete files or files at risk of obsolescence. The software additionally, provides specific solutions with high security requirements for those organizations that need to preserve automatically the integrity and accessibility of large, catalogued and searchable archives. As an easy to use tool, DURAFILE will facilitate the preservation of valuable information, and will provide a long term access to data for organizations and individual users.
DURAFILE will improve the competitiveness of the European economy in several ways:
1. Tapping into a huge market potential due to the vast potential user list and exponential growth of digital data.
2. First-mover advantage in comparison to non-European countries that are not using DURAFILE.
3. First Software solution for home users and small-medium size companies
Regarding the dissemination activities, the consortium created a dissemination plan in for spreading DURAFILE through publications, magazines, media, advertising and other events. As chairman of the Exploitation Board, Mr Miquel Montaner (EASY), coordinated all the activities encompassing any transfer of knowledge outside the consortium. All the partners took an active role in discussing and deciding the content of the material to be published in order to ensure that initially no confidential information was disclosed on potentially marketable aspect of the project. The consortium focused on explain the existing problem that DURAFILE is going to solve. The Consortium detected that few companies and almost none person is concerned about the problems of digital obsolescence and that there is no similar commercial product. This fact represents and advantage in terms of exploitation in the sense that the SMEs can be the first in positioning in the digital preservation market. However, it also represents a major difficulty, because the SMEs have not only to sell the product, but to create the need among the potential clients.
The main dissemination activities during the project were:
In order to start with the marketing activities, the Consortium prepared during the second period two market-oriented videos addressed on the main target clients (professional users and home users).
Moreover, the Consortium has dissemination material in order to be shown to the potential clients:
• A power point presentation of the project.
• An updated website of the project (http://www.durafile.eu)
• 3 Video Clips: one for the project video clip prepared during the first period, a second one for the corporate users’ video clip, and the third one for home users video clip.
• A GUI Tutorial
The consortium carried out specific dissemination actions of the DURAFILE project in the media and some conferences related to the Digital Preservation section. The main dissemination events were:
• December 2013. DURAFILE project promotion at La Xarxa Radio Station (Spain).
• February 2015. DURAFILE project presentation at the Digital Memory VI conference (Estonia)
• May 2015. DURAFILE project presentation at the International Archiving conference (Spain).
• August 2015. DURAFILE paper on the International Conference on Digital Preservation (Spain).
List of Websites: