Periodic Reporting for period 1 - LEAN MOBILE MAPPING (EUROPES FIRST SCALABLE MOBILE MAPPING SOLUTION WHICH EMPOWERS ALL BUSINESSES AND INSTITUTIONS TO ACCESS ESSENTIAL GEO-SPATIAL DATA)
Reporting period: 2018-12-01 to 2019-03-31
Summary of the context and overall objectives of the project
Over €2.4tn in revenues are lost annually due to poor data quality with 85% of all business data either being obsolete or of unknown value. 80% of all data owned by businesses and institutions is in the form of images or videos which are notoriously more difficult to analyze than simple text. All data needs to be updated within 1-24 months to remain useful but due to the high processing costs, businesses are unable to keep their data relevant. Current solutions to this problem are either offering (1) costly cameras and require the customer to process data manually or (2) a full- service which does all processing and data collection at a high service cost. We have realized that the key to providing a sustainable, low-cost and high-quality solution requires automation of the labour-intensive processing work. With LEAN MOBILE MAPPING, we provide Artificial Intelligence powered object recognition in combination with a small and high-performance camera system that can be placed on any vehicle, boat, train or other moving platform to perform location-based asset management.
Lean Mobile Mapping was developed as a Minimum Viable Product for this Feasibility Study in collaboration with a customer of the Netherlands who was able to perform automated scanning of bus stops. This is a project this customer has to do regularly and it used to take 4 times as long as it did now.
We have been able to investigate the benefits of this new method not only to our existing customers but we have also discovered new industry verticals benefiting from the availability of Lean Mobile Mapping. Furthermore, we have made first market research in two new countries (Italy and Poland) and are now in the process of selling to two customers in these countries. In two other countries we are in advanced talks with customers and expect to start sales talks soon.
We have not only made progress in assessing the market interest in Lean Mobile Mapping but have also been able to assess our business model and product pricing based on our market research.
Based on our findings we have come to encouraging conclusions:
• there is a validated and clear market demand for Lean Mobile Mapping
• we are able to offer Lean Mobile Mapping at a price point that enables a sustainable business
• there is growth potential across industry verticals and in different European countries
Lean Mobile Mapping was developed as a Minimum Viable Product for this Feasibility Study in collaboration with a customer of the Netherlands who was able to perform automated scanning of bus stops. This is a project this customer has to do regularly and it used to take 4 times as long as it did now.
We have been able to investigate the benefits of this new method not only to our existing customers but we have also discovered new industry verticals benefiting from the availability of Lean Mobile Mapping. Furthermore, we have made first market research in two new countries (Italy and Poland) and are now in the process of selling to two customers in these countries. In two other countries we are in advanced talks with customers and expect to start sales talks soon.
We have not only made progress in assessing the market interest in Lean Mobile Mapping but have also been able to assess our business model and product pricing based on our market research.
Based on our findings we have come to encouraging conclusions:
• there is a validated and clear market demand for Lean Mobile Mapping
• we are able to offer Lean Mobile Mapping at a price point that enables a sustainable business
• there is growth potential across industry verticals and in different European countries
Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far
Technical feasibility study:
• Software Scalability:
A key technical requirement of Leam Mobile Mapping is the ability to serve customers within a wide variety of needs in terms of the size of the areas they need to map. Some orginsations have fairly small areas they wish to map, while others need maps of very large areas with thousands of kilometers of roads, and anything in between. This has major implications on the requirments of scaling not just data storage but also data accessibility and processing power. Since the bulk of the data produced is video, data amounts are considerable. One of the crucial abilities of our software is object recognition. This means that a vast amount of data of varying size per customer and per mapping project needs to be processed quickly due to the workflow of our customers: (workflow schedule). Such workflow can only ever be achieved by flexible and scalable cloud computing.
We have contacted the two market leaders in scalable cloud computing, Amazon AWS and Google Cloud Platform. Both were quickly convinced of the potential value of Lean Mobile Mapping and have offered us free credits to try out their cloud platforms. We are now at the point of applying for the Google Surge program, which is an upgrade for carefully selected high potential tech startups.
We have also been selected as a leading tech startup by Nvidia, the market leader in GPU technology. This is a nice recognition of the value promised by Lean Mobile Mapping.
Thanks to the free cloud credits, we have been able to run scalability tests of our software. We have done this by processing our data in increasing parallel runs. We have over 5TB of data that is analysed by GPUs in the cloud. Processing this amount with a single GPU (we use Nvidia Tesla K80 GPUs) would take several months. We have started parallel processing with 20 GPUs, which would still take a few months with this amount of data. Then we have experimented with 50, 100, 200 and eventually with as much as 500 GPUs. Setting this up is a bit of work, but the result has been worth is: there is no reason to worry about scalability.
To put these numbers in perspective, this is over 18.000 km of roads. In other words, our system has proven to be able to process at least this amount of mapping simultaneously. The advantage of using a third party cloud provider is precisely this scalability. We can configure according to need and scale back in less intensive periods.
This affords us enough confidence about the performance of our system at scale.
Scalability of our system's performance is crucial because customers expect results from their mapping the next day. The workflow is such that they cover an area to map and then upload the data to the cloud after their mapping trip. This usually happens at night. The result would then be available the morning after.
• AI algorithm and additional features:
As stated, object recognition is a central ability of our product. Our customers use our mobile camera system to record video, from which our software then needs to extract the information that the customer is interested in . This can be different for each customer and even vary per project. We initially anticipated the need for a single image recognition algorithm, but have learned that there is no ""one size fits all"" approach so there are several algorithms to be developed. Another key finding is that just object recognition is not sufficient. A key feature of maps is the accurate positioning of objects. This is important because of several reasons:
• professionals need accurate positions of the objects of interest in order to execute their operations precisely. For example, one of the object types we map are traffic signs. If the accuracy of traffic sign positioning is off by a few meters, there is a significant risk that you position them in the wrong street. This would make the map useless for traffic management professionals
• objects need an unique identifier. This is especially important in case there are several similar objects near each other. For example, one of our customers has mapped bollards on their quay walls, which come in pairs. The only way to uniquely identify such objects is their unique geographical position. Since one of the objectives of this customer is to do a regular assessment of the quality of their quay wall assets, their should be no confusion about the unique ID of objects. Therefore, it is adamant that the software is capable of positioning objects within very close tolerance (~10 cm). In traditional mobile mapping, this is achieved through the use of lidar technology. However, lidar technology is too expensive for our use. A central element in our value proposition is that our hardware is a simple, affordable camera that can be used by non-trained operators. Lean Mobile Mapping uses just video so there has to be a positioning mechanism. This can be achieved through a combination of several algorithms:
• triangulation: this is a well know method, mimicking nature. Indeed, it is the natural way us humans are able to see depth. Because we have two eyes, with a slight distance between them, we see objects from a slightly different angle and are thus seeing from two perspectives. Our brain combines these two images and enables us to estimate distance. The same effect is often used in machine vision, using two cameras positioned slightly apart to recreate the effect. This is called stereovision. However, the philosophy of Lean Mobile Mapping is to minimize hardware complexity and cost. Still, the same effect can be achieved with triangulation. What is needed is at least two images that are taken from a different position. This is exactly what our camera does: it records video, so there are several images. And it is moving, so these images are taken from different positions. As the vehicle is moving, video frames of objects are created from different positions. These different points of view form a triangle, the dimensions of which can be calculated. There is, however, a key requirement for this principle to work and that is that we know the exact geoposition of the camera and the direction it is facing. Therefore the hardware will have to be improved, as currently the accuracy of the GNSS receiver is inadequate for this level of positioning.
• depth mapping: we have investigated and tried to implement a depth estimation algorithm. While this certainly has promising merit, the algorithm needs considerable work to be optimised for our specific hardware. This is because the quality of such algorithm depends heavily on the quality and amount of training data available. We have over the last months created sufficient amounts of data, but it is recommended for several reasons to upgrade the hardware to produce higher quality video. One implication of this upgrade, however, is that the algorithm will have to be retrained with new video data. Therefore we have decided not to implement this depth estimation algorithm in this phase beyond some initial proof of concept. This proof of concept demonstrates that this depth estimation algorithm is not only feasible, but is likely to support triangulation to improve positioning accuracy.
• anonymisation: Lean Mobile Mapping needs to comply to privacy regulations and concerns. One of the features to achieve this is anonymising the video. We achieve this by automatically blurring faces of persons and car license plates. Since privay is a genuine concern legally and ethically, we have given this feature high priority. This is now implemented. We have achieved this by developing an algorithm that is able to detect persons and cars and then adds Gaussian blur to faces and license plates. This happens in the background, before the data is accessible by humans.
• Route optimisation:
Due to time constraints, route optimisation is not completely finished. The core is there, but it still needs manual work. However, this will likely be finalised in the near future. It is a feature that is definitely necessary, but not as urgent or crucial as the AI part
• Coaching: during the coaching, many aspects of Lean Mobile Mapping were challenged and sometimes improved. For example, a better pricing model was reached. Also the need to focus on just a few industry verticals instead of many became clear.
• Software Scalability:
A key technical requirement of Leam Mobile Mapping is the ability to serve customers within a wide variety of needs in terms of the size of the areas they need to map. Some orginsations have fairly small areas they wish to map, while others need maps of very large areas with thousands of kilometers of roads, and anything in between. This has major implications on the requirments of scaling not just data storage but also data accessibility and processing power. Since the bulk of the data produced is video, data amounts are considerable. One of the crucial abilities of our software is object recognition. This means that a vast amount of data of varying size per customer and per mapping project needs to be processed quickly due to the workflow of our customers: (workflow schedule). Such workflow can only ever be achieved by flexible and scalable cloud computing.
We have contacted the two market leaders in scalable cloud computing, Amazon AWS and Google Cloud Platform. Both were quickly convinced of the potential value of Lean Mobile Mapping and have offered us free credits to try out their cloud platforms. We are now at the point of applying for the Google Surge program, which is an upgrade for carefully selected high potential tech startups.
We have also been selected as a leading tech startup by Nvidia, the market leader in GPU technology. This is a nice recognition of the value promised by Lean Mobile Mapping.
Thanks to the free cloud credits, we have been able to run scalability tests of our software. We have done this by processing our data in increasing parallel runs. We have over 5TB of data that is analysed by GPUs in the cloud. Processing this amount with a single GPU (we use Nvidia Tesla K80 GPUs) would take several months. We have started parallel processing with 20 GPUs, which would still take a few months with this amount of data. Then we have experimented with 50, 100, 200 and eventually with as much as 500 GPUs. Setting this up is a bit of work, but the result has been worth is: there is no reason to worry about scalability.
To put these numbers in perspective, this is over 18.000 km of roads. In other words, our system has proven to be able to process at least this amount of mapping simultaneously. The advantage of using a third party cloud provider is precisely this scalability. We can configure according to need and scale back in less intensive periods.
This affords us enough confidence about the performance of our system at scale.
Scalability of our system's performance is crucial because customers expect results from their mapping the next day. The workflow is such that they cover an area to map and then upload the data to the cloud after their mapping trip. This usually happens at night. The result would then be available the morning after.
• AI algorithm and additional features:
As stated, object recognition is a central ability of our product. Our customers use our mobile camera system to record video, from which our software then needs to extract the information that the customer is interested in . This can be different for each customer and even vary per project. We initially anticipated the need for a single image recognition algorithm, but have learned that there is no ""one size fits all"" approach so there are several algorithms to be developed. Another key finding is that just object recognition is not sufficient. A key feature of maps is the accurate positioning of objects. This is important because of several reasons:
• professionals need accurate positions of the objects of interest in order to execute their operations precisely. For example, one of the object types we map are traffic signs. If the accuracy of traffic sign positioning is off by a few meters, there is a significant risk that you position them in the wrong street. This would make the map useless for traffic management professionals
• objects need an unique identifier. This is especially important in case there are several similar objects near each other. For example, one of our customers has mapped bollards on their quay walls, which come in pairs. The only way to uniquely identify such objects is their unique geographical position. Since one of the objectives of this customer is to do a regular assessment of the quality of their quay wall assets, their should be no confusion about the unique ID of objects. Therefore, it is adamant that the software is capable of positioning objects within very close tolerance (~10 cm). In traditional mobile mapping, this is achieved through the use of lidar technology. However, lidar technology is too expensive for our use. A central element in our value proposition is that our hardware is a simple, affordable camera that can be used by non-trained operators. Lean Mobile Mapping uses just video so there has to be a positioning mechanism. This can be achieved through a combination of several algorithms:
• triangulation: this is a well know method, mimicking nature. Indeed, it is the natural way us humans are able to see depth. Because we have two eyes, with a slight distance between them, we see objects from a slightly different angle and are thus seeing from two perspectives. Our brain combines these two images and enables us to estimate distance. The same effect is often used in machine vision, using two cameras positioned slightly apart to recreate the effect. This is called stereovision. However, the philosophy of Lean Mobile Mapping is to minimize hardware complexity and cost. Still, the same effect can be achieved with triangulation. What is needed is at least two images that are taken from a different position. This is exactly what our camera does: it records video, so there are several images. And it is moving, so these images are taken from different positions. As the vehicle is moving, video frames of objects are created from different positions. These different points of view form a triangle, the dimensions of which can be calculated. There is, however, a key requirement for this principle to work and that is that we know the exact geoposition of the camera and the direction it is facing. Therefore the hardware will have to be improved, as currently the accuracy of the GNSS receiver is inadequate for this level of positioning.
• depth mapping: we have investigated and tried to implement a depth estimation algorithm. While this certainly has promising merit, the algorithm needs considerable work to be optimised for our specific hardware. This is because the quality of such algorithm depends heavily on the quality and amount of training data available. We have over the last months created sufficient amounts of data, but it is recommended for several reasons to upgrade the hardware to produce higher quality video. One implication of this upgrade, however, is that the algorithm will have to be retrained with new video data. Therefore we have decided not to implement this depth estimation algorithm in this phase beyond some initial proof of concept. This proof of concept demonstrates that this depth estimation algorithm is not only feasible, but is likely to support triangulation to improve positioning accuracy.
• anonymisation: Lean Mobile Mapping needs to comply to privacy regulations and concerns. One of the features to achieve this is anonymising the video. We achieve this by automatically blurring faces of persons and car license plates. Since privay is a genuine concern legally and ethically, we have given this feature high priority. This is now implemented. We have achieved this by developing an algorithm that is able to detect persons and cars and then adds Gaussian blur to faces and license plates. This happens in the background, before the data is accessible by humans.
• Route optimisation:
Due to time constraints, route optimisation is not completely finished. The core is there, but it still needs manual work. However, this will likely be finalised in the near future. It is a feature that is definitely necessary, but not as urgent or crucial as the AI part
• Coaching: during the coaching, many aspects of Lean Mobile Mapping were challenged and sometimes improved. For example, a better pricing model was reached. Also the need to focus on just a few industry verticals instead of many became clear.
Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)
Lean Mobile Mapping has an important societal impact for a number of reasons:
• the state of the art in mobile mapping is driving with a customised vehicle equipped with very complex and expensive mapping equipment. Due of this, the quality of mapping data is suffering because generally map cannot be updated often enough so they are often outdated. Another consequence is that many organisations that need map data simply don't have access to it. Lean Mobile Mapping disrupts this state of the art. We have customers who have been using pen and paper in the field to create very low quality information, but it was the only option they had. Thanks to Lean Mobile Mapping these organisations now have the ability to create high quality maps and keep them up to date
• For example, we have encountered that many medium sized ports would be able to improve their operations considerably with digital maps, which they currently mostly don't have due to budget constraints. Having access to digital mapping through Lean Mobile Mapping has an important economical impact
• Lean Mobile Mapping was used in a sustainable mobility project by the city of Roeselare, Belgium, that won a Smart City Award (https://www.roeselare.be/nieuws/roeselare-wint-agoria-smart-city-award-met-project-verkeersveiligheid - in Dutch). Since mobility is a huge challenge in many European cities and carries an enormous opportunity cost and societal loss, having better data for mobility is bound to have a major impact. One of the issues is that road authorities generally don't have adequate information to base optimal mobility policies on. For example, one would assume that authorities have good databases and maps of traffic signs. However, most don't. Using Lean Mobile Mapping, it becomes very feasible to not only create an accurate high quality database of traffic signs, but also to keep it up-to-date. This is important as the public space evolves over time. In order to design better mobility, authorities need better information and Lean Mobile Mapping can play a pivotal role in this
• Another example is a study of bus stops in the province of Utrecht, the Netherlands, by our customer. The aim was to map the use level of bus stops in order to help infrastructure investment decisions. Using Lean Mobile Mapping, the number of parked bicycles at bus stops was counted - in the Netherlands, where cycling is especially popular, this is an excellent proxy to the number of commuters at a stop. This kind of information helps better allocate budgets, giving better value for money and better customer satisfaction
• The AI of Lean Mobile Mapping takes away the complexity traditionally associated with mobile mapping. Instead of just trained operators, virtually anyone can use the Lean Mobile Mapping system. This creates opportunities for jobs for less skilled people
• the state of the art in mobile mapping is driving with a customised vehicle equipped with very complex and expensive mapping equipment. Due of this, the quality of mapping data is suffering because generally map cannot be updated often enough so they are often outdated. Another consequence is that many organisations that need map data simply don't have access to it. Lean Mobile Mapping disrupts this state of the art. We have customers who have been using pen and paper in the field to create very low quality information, but it was the only option they had. Thanks to Lean Mobile Mapping these organisations now have the ability to create high quality maps and keep them up to date
• For example, we have encountered that many medium sized ports would be able to improve their operations considerably with digital maps, which they currently mostly don't have due to budget constraints. Having access to digital mapping through Lean Mobile Mapping has an important economical impact
• Lean Mobile Mapping was used in a sustainable mobility project by the city of Roeselare, Belgium, that won a Smart City Award (https://www.roeselare.be/nieuws/roeselare-wint-agoria-smart-city-award-met-project-verkeersveiligheid - in Dutch). Since mobility is a huge challenge in many European cities and carries an enormous opportunity cost and societal loss, having better data for mobility is bound to have a major impact. One of the issues is that road authorities generally don't have adequate information to base optimal mobility policies on. For example, one would assume that authorities have good databases and maps of traffic signs. However, most don't. Using Lean Mobile Mapping, it becomes very feasible to not only create an accurate high quality database of traffic signs, but also to keep it up-to-date. This is important as the public space evolves over time. In order to design better mobility, authorities need better information and Lean Mobile Mapping can play a pivotal role in this
• Another example is a study of bus stops in the province of Utrecht, the Netherlands, by our customer. The aim was to map the use level of bus stops in order to help infrastructure investment decisions. Using Lean Mobile Mapping, the number of parked bicycles at bus stops was counted - in the Netherlands, where cycling is especially popular, this is an excellent proxy to the number of commuters at a stop. This kind of information helps better allocate budgets, giving better value for money and better customer satisfaction
• The AI of Lean Mobile Mapping takes away the complexity traditionally associated with mobile mapping. Instead of just trained operators, virtually anyone can use the Lean Mobile Mapping system. This creates opportunities for jobs for less skilled people