Periodic Reporting for period 1 - DUF (Deep Learning UAV Networks for Autonomous Forest Firefighting)
Periodo di rendicontazione: 2017-04-01 al 2019-03-31
Thousands of hectares of forest lands are lost to wildfires every year. Utilization of Unmanned Aerial Vehicles (UAVs) is an efficient tool for fighting fires, however the state-of-the-art techniques lack in ability to predict fire spread direction and coordinate multiple UAVs to suppress the fire under limited communication. DUF project aims to apply powerful tools from artificial intelligence domain to UAV firefighting problem, creating an innovative solution for autonomous firefighting, which reduces the amount of lands lost to fires (see Figure 1). DUF uses the multiagent deep learning techniques for estimating the fire spread direction obtained from measurements from UAVs. After the model is learned, a novel graphical encoding based deep reinforcement learning algorithm is utilized to coordinate UAV actions for suppressing the fire. The algorithm development and simulations are conducted at Istanbul Technical University (ITU) Aerospace Research Center (ARC).
Development of Stochastic Fire Spread Model:
A parametric forest fire simulation model was developed combining data and previous work from literature. The model is capable of simulating forest fire based on initial condition of the fire, distribution of trees/bushes in the forest, overall topology of the forest including flammable and inflammable objects and direction of wind. The model also has the capability to simulate heterogeneous wind conditions and fire jumps.
Development of Multi-UAV Learning Algorithms:
A novel supervised deep learning model was developed for learning the fire spread dynamics from observations obtained from the model. The deep neural network model utilizes both convolutional and recurrent layers to predict the fire spread from the current observations obtained from the model. At first stage, each UAV locally runs the developed algorithm to predict fire spread for their local regions. Next, a transfer learning based algorithm is developed for fusing the local models learned by each UAV. The transfer learning algorithm combines the neural network models learned by each UAV and merges them to a single improved model, which can be used by all UAVs for improved prediction quality. Simulation results show that the developed algorithm improves the learning significantly compared to approaches where agents learn independent from each other.
Development of Multi-UAV Planning Algorithms:
A novel graph based deep learning algorithm is developed for solving large scale multi-UAV fire suppressing problems. Algorithm first learns a graph structure that explains the cooperation structure of the UAVs based on the current policy they are executing. Next, a graph convolutional network is trained from the state transitions and a greedy Q-learning based algorithm is used for updating the policy. The algorithm is tested on a combination of different fire scenarios with different forest topologies. Simulation results show that when combined with the multiagent learning algorithm, the developed planning algorithm significantly outperforms the current deep reinforcement learning approaches in terms of percentage of forest area saved from fire.
A parametric forest fire simulation model was developed combining data and previous work from literature. The model is capable of simulating forest fire based on initial condition of the fire, distribution of trees/bushes in the forest, overall topology of the forest including flammable and inflammable objects and direction of wind. The model also has the capability to simulate heterogeneous wind conditions and fire jumps.
Development of Multi-UAV Learning Algorithms:
A novel supervised deep learning model was developed for learning the fire spread dynamics from observations obtained from the model. The deep neural network model utilizes both convolutional and recurrent layers to predict the fire spread from the current observations obtained from the model. At first stage, each UAV locally runs the developed algorithm to predict fire spread for their local regions. Next, a transfer learning based algorithm is developed for fusing the local models learned by each UAV. The transfer learning algorithm combines the neural network models learned by each UAV and merges them to a single improved model, which can be used by all UAVs for improved prediction quality. Simulation results show that the developed algorithm improves the learning significantly compared to approaches where agents learn independent from each other.
Development of Multi-UAV Planning Algorithms:
A novel graph based deep learning algorithm is developed for solving large scale multi-UAV fire suppressing problems. Algorithm first learns a graph structure that explains the cooperation structure of the UAVs based on the current policy they are executing. Next, a graph convolutional network is trained from the state transitions and a greedy Q-learning based algorithm is used for updating the policy. The algorithm is tested on a combination of different fire scenarios with different forest topologies. Simulation results show that when combined with the multiagent learning algorithm, the developed planning algorithm significantly outperforms the current deep reinforcement learning approaches in terms of percentage of forest area saved from fire.
Progress beyond state of the art:
Multiagent planning and learning algorithms developed in the project outperformed state of the art deep reinforcement learning algorithms in terms of sample efficiency and percentage of the forest saved from the fire. This performance is achieved by:
- Using transfer learning methods to accelerate the model learning process, hence learning the fire spread model by using smaller number of samples.
- Using graphical models to decompose the multi-UAV problem into a sequence of smaller problems, hence accelerating the rate of learning and in turn improving planning performance.
Potential Impact:
The algorithms developed in the project has the potential to predict fire spread quickly from UAVs, hence if deployed in real life systems, they have the potential to increase the efficiency of forest firefighting operations.
In terms of wider impact, although the algorithms originally developed for multi-UAV forest firefighting, they can be easily extended to other multi-robotic planning domains, such as surveillance, planetary exploration and infrastructure inspection.
Multiagent planning and learning algorithms developed in the project outperformed state of the art deep reinforcement learning algorithms in terms of sample efficiency and percentage of the forest saved from the fire. This performance is achieved by:
- Using transfer learning methods to accelerate the model learning process, hence learning the fire spread model by using smaller number of samples.
- Using graphical models to decompose the multi-UAV problem into a sequence of smaller problems, hence accelerating the rate of learning and in turn improving planning performance.
Potential Impact:
The algorithms developed in the project has the potential to predict fire spread quickly from UAVs, hence if deployed in real life systems, they have the potential to increase the efficiency of forest firefighting operations.
In terms of wider impact, although the algorithms originally developed for multi-UAV forest firefighting, they can be easily extended to other multi-robotic planning domains, such as surveillance, planetary exploration and infrastructure inspection.