The THINKlab has developed a new digital drone visualisation platform which has the potential for changing the way users can respond to disasters more effectively and for supporting post disaster reconstruction efforts.
The digital drone visualisation platform will mean that stakeholders are able to interpret data more quickly which will strengthen their decision-making and increase their intelligence when responding to disaster situations. This environment will allow emergency operation centres to receive live videos and construct 3D representations of the disaster areas to establish a situational awareness on the ground.
By linking this drone environment with the MOBILISE 3.0 digital disaster management platform, decision-makers can extract further information about the built environment, infrastructures and communities to establish a more holistic view of the situation and ultimately save lives. The digital drone visualisation platform can also be used for other applications such as urban planning and construction site monitoring. By using the drone visualisation platform integrated with the city information model provided through the MOBILISE 3.0 platform, planners can create up-to-date city models in a short period of time to support urban regeneration projects.
THINKlab Co-ordinator Simon Hadfield spoke to its developers Mikhail Polshaw (VR Software Engineer) and Adam Sinclair (VR Content Creator) to understand how this programme is changing the way the THINKlab’s research teams are working with their partners.
You’ve been working on the drone visualisation platform over the last year and now we’re starting to really see its potential. Can you explain what this platform is?
Mike: The drone visualisation platform is a tool which helps the user or stakeholder to understand what is happening on the ground. It allows the user to visualise live video data from drones in an accurate 3D environment. It can be used by itself or integrated with the MOBILISE 3.0 platform so the drone data can be viewed alongside other city information that has been uploaded to the MOBILISE platform. Using the live data and MOBILISE data alongside each other, you can quickly identify and understand risk situations and plan interventions. Some of these risks might arise from flooding and landslides. As well as live drone footage, the data is recorded so as part of an ongoing process, as newer data comes in, you can easily compare what has changed.
How does the drone visualisation platform work?
Mike: We have developed the platform initially using the DJI series of drones who are a world leader in aerial photography systems. However, if the drone position can be accurately queried any type of drone can be used. We currently use positional data and the live video feed, though any extra sensors on a drone could also be included. These extra sensors might include further data capture elements such as wind speed, infrared or depth. The drone professional can responsibly fly the drone across an area where something has happened, or something might be happening. The data which is captured can then be overlaid directly with the data in the MOBILISE platform.
You developed the drone visualisation platform using the Unity Gaming Platform, how did you go about building it?
Adam: We used Unity which is a market-leading platform for creating interactive real-time 3D content. It is being used across multiple industries such as gaming, transport, engineering and architecture, to develop and advance 3D visualisation. When we are using this piece of software, we can display 3D representations of real-world terrain in higher detail than on the web platform which MOBILISE can run. To generate a 3D mesh terrain, we are using Agisoft Metashape to process the image data collected from the drones. Agisoft Metashape is photogrammetry software that allows you to create 3D models from a series of digital images. We have implemented an automated process where a video or selection of images, captured from a drone, can be converted into an accurate geo-located 3D mesh using photogrammetry.
Using the drone visualisation platform, the user can very quickly visualise, compare and modify drone scanned terrains and meshes. A user can very easily import a processed terrain (from Agisoft), integrate it with an existing project and compare this with previous versions to understand and take measurements. This will allow an ongoing validation and modification process. When it is connected to the MOBILISE platform, it allows a vast amount of data to be stored in the database which is displayed on the high-quality terrain meshes.
What can the user expect from the interface?
Adam: We have developed an interface that functions similarly to the MOBILISE platform with additional functionality for displaying high-quality 3D terrain data. The interface has been designed for use with a mouse and keyboard as well as with a touchscreen which our partners are currently using in several Disaster Management Centres. Users can overlay data onto the terrain, click on specific buildings and landmarks to understand additional information, and display and manage multiple drone flights. This data has been captured in the past or is being taken live, so in real time. Users can follow drone flight paths, zoom in to see the drone’s camera view or project what the drone is currently seeing onto 3D terrain. Using the interface, you can compare what a drone can see with terrain that has been captured and processed previously. This can then be used to view the result of a landslide, flood or other areas such as progressing construction work in a specific area. We are implementing additional tools through the feedback of our stakeholders which include the ability to check terrain height, measure distances and check gradients.
Mike: In the MOBILISE platform, users can upload geo-coded data layers to view building types, roads, hazard areas, flood levels, etc. The drone visualisation platform has access to this data and as the drone data is also geo-coded, the various data layers can be queried directly. This allows the user to isolate specific buildings or areas that are determined a high risk or are more vulnerable directly from the drone visualisation interface. Bringing the two interfaces together requires accessing the data in the MOBILISE web interface directly from the drone visualisation platform. When the two are synchronised, you can drill down on specific locations to study the built environment and see where the infrastructure is and how it might be affected by hazards and risks. One of the major technological advances with MOBILISE 3.0 is the integration with the drone visualisation platform. Building on the previous MOBILISE platform (which collected data to create visual narratives), when the drone visualisation platform is used with MOBILISE 3.0, it will provide stakeholders with a deeper understanding of vulnerability and recommend mitigation measures.
What makes the drone visualisation platform unique and what are the user benefits?
Adam: Used with MOBILISE 3.0, the drone visualisation platform will have access to all the new features available with the programme. This will include visualising the data and a more intelligent view of what that data means. With MOBILISE the data can be mined, processed and presented in a fashion that is most useful for the task at hand. While there are other applications which support the use of drones, many of these are stand-alone applications which don’t speak to each other. What is unique about the platform is that the user can see the drone and plot its course, visualise and work with GIS data (Geographical Information System, a framework to capture and analyse spatial and geographic data) and other datasets altogether in one easy to use interface. Utilising a common set of user interfaces, the different roles in any situation can easily be viewed with the data that is relevant to them, giving a better understanding. Used with MOBILISE, the user has immediate access to information, whereas previously it would take several analysts to come together to understand different data sets.
The new digital drone visualisation platform has been developed as part of the MOBILISE project which is funded by the Global Challenges Research Fund (GCRF) and The Engineering and Physical Sciences Research Council (EPSRC). The MOBILISE project focuses on LMIC communities in Sri Lanka, Malaysia and Pakistan who are regularly affected by natural disasters such as flooding and landslides.
The drone data visualisation platform supports agencies in their post-disaster reconstruction activities. The MOBILISE project team has been working with Sri Lanka’s National Building Research Organisation (NBRO) and the Sri Lanka Disaster Management Centre to support their disaster investigations and reconstruction activities to build back better. Read more here.