Access to Open Imagery: Innovation

In 2014 the Humanitarian OpenStreetMap Team (HOT) was awarded a development grant by HIF and after 10 months of intense design and coding, OpenAerialMap (OAM) is finally here! Anyone can now go to www.openaerialmap.org and find openly licensed Earth imagery that can be publicly used for any purpose, and in particular, to support humanitarian mapping with OpenStreetMap. A single place to find open imagery had not existed before OAM was created. Those wanting to share image data and mappers looking for imagery can now easily meet, thanks to this innovative system.

The idea behind OpenAerialMap

The idea of creating OpenAerialMap dates back to 2007 when satellite imagery providers started making their data freely available for disaster response mapping. It was suddenly clear that the humanitarian mapping community was not able to easily ingest and organize all this wealth of information. It took several years of brainstorming and prototyping to come to the design that HOT proposed to HIF. This included creating an open source set of tools that would provide functions for hosting, uploading, sharing, searching, filtering, displaying, downloading and using imagery data. All these features have now been implemented and the software source code is available in Github. OAM is currently hosted in HOT’s computing infrastructure, but its code is available for anyone wanting to replicate the same system in local contexts or through cloud service platforms.

HOT launched a call for developers in February and among excellent applications, the Development Seed team led by Nate Smith was selected. Their experience in designing mapping user interfaces and developing open source tools was invaluable to building an attractive and effective OAM Catalog application. This consists of an interactive, easy-to-use map browser presented with a dynamically shaded grid that symbolizes density and availability of images across the world. Users can navigate the map and click on any cell to display footprints, overviews and details for each image. As illustrated in figure 1, additional filtering tools provide more control on what data is displayed on the map. For example, following a specific event is possible to show only imagery that was collected in the last 7 days over a specific location. All metadata and map service information is available to the users both visually and through an application programming interface (API). This provides computer based open and interoperable access to the entire OAM Catalog, allowing any developer to easily make use of the imagery and build applications that automatically retrieve its data.

The challenges and goals

One issue identified in early development was about costs of hosting and distributing imagery data, which usually consisting of very large multi-gigabyte files. It was clear that no single organization could afford these costs, therefore a solution was created called Open Imagery Network (OIN). As shown in figure 2 (view in full here), OIN is a federated network of highly available (always accessible and offering large bandwidth) imagery “buckets”. These may include public storage endpoints on cloud services like Amazon S3 or Microsoft Azure. All these repositories of individual files are paid for by OIN members, including original satellite image providers and foundations supporting humanitarian mapping, thus spreading the costs. OAM then provides API and browser access to the entire collection of images through a common OIN index file.

One important goal of OAM was to make imagery easily accessible in environments where limited bandwidth and computer resources wouldn’t allow downloading large individual files. A component called “OAM Server” was developed by Rob Emanuele at Azavea and Seth Fitzsimmons at Stamen to process batches of original images into tile map services (TMS). Similar to how imagery is offered in proprietary platforms such as Google Maps, large imagery mosaics are “chopped” into small and compressed tiles that can be easily displayed inside a Web browser application. This is particularly useful for mapping in OpenStreetMap where volunteers don’t usually have the required knowledge to work with large GIS files, but can instead easily overlay an image layer streaming directly from OAM and requesting exclusively small image tiles for the areas and zoom levels displayed on the map. Cutting large image mosaics into a TMS is a computing intensive process, which usually requires days of single workstation work. OAM developers decided to adopt a scalable processing system called Elastic Map Reduce (EMR) to quickly pool large numbers of cloud processing units to perform the tiling task in parallel and drastically reducing compute time. Thanks to the pay-as-you-go model available on cloud service providers like Amazon Web Services (AWS), these bursts of computing clusters only running for a short amount of time, makes the tiling process very cost effective and efficient.

The launch and beyond

Since the OpenAerialMap catalog was launched in beta a few months ago, interest in the system has been expressed by several communities, including the UAV Humanitarian Network (UAViators), which are now using OAM to share imagery collected with drones to map areas affected by natural disasters. In addition to finding the system very helpful for hosting imagery, UAViators suggested that OAM could one day extend to include three-dimensional information, such as 3D point clouds, also a very useful byproduct of drone imagery processing.

In addition to all the code and software, OAM is now a community of developers, volunteer mappers, drone pilots, geospatial professionals, and government stakeholders. There is growing interest by humanitarian organizations and satellite imagery providers to support OpenAerialMap and Open Imagery Network. Companies like Amazon have already committed funds to sustain the core operational functionalities for 2016, while a concept note has been drafted to create a international foundation of donors that would sponsor openly licensed imagery acquisition and hosting to support humanitarian and development projects. OAM and OIN have been designed to be easily scalable, thanks to the open source nature of their code and the distributed approach to hosting imagery. It is expected that the innovation introduced by OAM will be able to sustain and be adopted even without a centralized massive source of funding.

We are excited to see OpenAerialMap already used in support of humanitarian and development projects such as Ramani Huria - sponsored by the World Bank - for disaster risk reduction in Dar es Salaam; or post event imagery for recovery mapping in the Philippines. One of the greatest successes of the project was not only building and making the system available to anyone, but also raising awareness on the importance of openly licensed imagery. Geospatial and humanitarian organizations often work with different file formats and proprietary imagery data, which makes exchange of critical information more difficult. By promoting open standards and common processing workflows, the OAM community continues to educate both providers and users on the importance of open licenses, as a way to innovate humanitarian response practice.

;

We finally want to thank HIF for the generous contribution to making this project successful. Despite initial clear ideas and requirements, it was a continuous collaborative learning process. HOT decided to adopt an open approach to its design and development. We engaged beneficiaries from the beginning by hosting public weekly meetings and making all project communication transparent. The rapid prototyping and agile development method adopted by OAM developers was also a key successful aspect which we highly recommend for any project wanting to engage their communities of users. We, at HOT, are grateful for all the skilled people who contributed to the project so far and we look forward to wide adoption of OAM and exciting new feature ideas for the next round of development.