Earth observation (EO) data from ground, airborne, and space platforms and associated applications have the potential to, and already, provide insights into global policy frameworks including: the United Nations (UN) 2030 Agenda for Sustainable Development [1
], the UN System of Environmental Economic Accounting [2
], the Sendai Framework for Disaster Risk Reduction [3
] and the Paris Climate Agreement [4
]. EO data can support, validate, and augment traditional data inputs, including national statistics, administrative data, household survey data and census information. In addition, EO data contribute as a direct indicator to inform relevant goals and targets; help optimize surveys and other traditional data collection efforts; and support disaggregation of targets and indicators, where relevant, to ensure that no one is left behind. Today, petabytes of EO data and geospatial information, coupled with analytical methods and innovation in technology, and enabled by free and open data policies, are applied widely around the world to derive useful information about the drivers, pace, and associated impacts of change on Earth, as well as to inform policies and support decision making.
Significant work is still needed, however, to ensure that different types of end-users are harnessing the full potential of EO to address local challenges and assist with the monitoring and implementation of global agendas, such as the UN 2030 Agenda for Sustainable Development. Improvements are needed to overcome challenges such as: EO data accessibility and handling; EO data validity and fitness for purpose; integration of information from different data streams; and data continuity [5
]. Organizations such as the Group on Earth Observations (GEO) and the Committee on Earth Observation Satellites (CEOS) are working to reduce the barriers that are faced by end-users across multiple sectors and regions in accessing, analyzing, and integrating satellite-based and other sources of EO data into national processes and decision support systems. More and more, there is a recognized need for new ways of managing and providing easy access to the vast amounts of EO that is increasingly available, as well as for raising awareness about the value of the data and translating science into policy.
In recent years, there has been a global move towards satellite operators producing analysis-ready data, to reduce the work needed by users prior to exploiting and analyzing satellite data. For example, CEOS has led the creation of the CEOS Analysis Ready Data for Land (CARD4L) framework. This framework defines CARD4L data as “… satellite data that have been processed to a minimum set of requirements and organized into a form that allows immediate analysis with a minimum of additional user effort and interoperability both through time and with other datasets” [6
The CARD4L framework has provided a set of product family specifications (PFS) for surface reflectance [7
], surface temperature [8
], and radar backscatter [9
]. These specifications, while not prescriptive, provide both minimum and target thresholds for general metadata, per-pixel metadata, radiometric and atmospheric corrections, and geometric corrections.
Satellite operators such as the United States Geological Survey (USGS) are now in the process of using these PFS to produce global collections of CARD4L compliant EO data. This transition to easily accessible CARD4L compliant data provides an incredible opportunity for EO data to be more impactful across a wide range of global challenges. However, the sheer amount of data that is now, or will soon be, available for use demands that we move away from the historical approach of users downloading data and local processing toward “processing into high performance computing data centers (e.g., Google Earth Engine, planet-API, National Computing Infrastructure in Australia, DigitalGlobe DGBX platform) using Big Data processing tools…along the lines of moving the algorithms to the data not the data to the algorithms” [10
]. Bringing together data, analytical methods, infrastructure, and application insights is essential to promote and accelerate social, economic, and environmental sustainability.
A range of software (open source and proprietary), tools, and analysis platforms exist for accessing, storing, processing, and facilitating the use of EO to derive insights and for societal applications. Cloud computing has had a tremendous effect on the emergence of computational infrastructure designed to provide EO analysis capabilities such as Google Earth Engine [11
], the Copernicus Data and Information Access Services [12
] and a range of other platforms provided via Amazon Web Services, Microsoft Azure (Layerscape), etc. Before choosing a software product, tool or analysis platform, end-users—including national statistical agencies, line ministries, and national mapping agencies, among other stakeholders—need to take into account their local needs including governance requirements, institutional capacity, geospatial analytics expertise, associated costs, and sustainability of the respective tool(s) or EO analysis platforms.
Countries such as Australia, Mexico, Switzerland, and Tanzania either have adopted or are in the process of adopting an open source solution, the Open Data Cube (ODC) [13
], to enable them to integrate insights from Earth observation data into their national policy and information systems. The ODC builds on the work of the Australian Geoscience Data Cube [15
] and seeks to increase the value and impact of global EO data by providing an open and freely accessible exploitation architecture, while fostering a community of cooperation that promotes open EO data, reuse of algorithms, and related information usage and sharing for the benefit of society.
The open source nature of the ODC was an important factor in this tool being selected by Australia, Mexico, Switzerland, and Tanzania. However, the other critical factor is the ODC’s ability to be implemented on diverse computational infrastructures ranging from national supercomputing facilities such as Australia’s National Computational Infrastructure, through to numerous commercial cloud infrastructures. Together, this combination of open source software and infrastructure flexibility has enabled the establishment of sovereign, operational capabilities that can be controlled and managed in-country. This is critical in order to build trust that the information products being generated can be both sustained and relied upon for use in a wide variety of policy problems.
Satellite missions will continue to provide increasingly larger volumes of free and open analysis-ready data for global users. With recent advances in the global provision of analysis-ready data and proven and innovative open source data technology solutions such as the Open Data Cube (ODC), global users now have the unprecedented ability to routinely utilize satellite data for national policy and decision-making needs.
The examples shown in this paper have demonstrated that governments have the need and desire to use satellite data to tangibly improve their management of natural resources and policies that support sustainable development. One of the reasons that ODC has been successful at enabling this is that it is an open source and scalable architecture; it allows countries to establish and operate their own sovereign analysis capability. Countries are able to control the quality and timeliness of their analyses and rely upon their own operational capability to underpin regulatory and official reporting processes.
However, great effort is needed for deploying an operational ODC at a national level. The vast amounts of data that need to be integrated and managed means that operational deployments need to have access to a wide range of system engineering skills and high-performance computational infrastructure. Consequently, there is an emerging trend to move from sovereign ODC deployments to larger scale regional centres that support sovereign ODCs. For example, by the end of 2019, the ARDC will merge into a larger regional cube including data from an ever growing range of satellites, including Landsat and Sentinel-2, which will cover the entire continent of Africa. This initiative is called Digital Earth Africa and serves as an example of a vision to create many regional data cubes using the ODC infrastructure.
Regional-scale data cubes are more manageable in terms governance and institutional arrangements, whereas a full global data cube would be impractical to implement and manage effectively. In addition, these regional cubes allow users to address transboundary topics that otherwise would not be possible with individual country-level data cubes. In this matter, standardization is key; it is essential that data and processes are consistent and measurements regard similar criteria. In the future, a set of regional data cubes could share technical approaches and application algorithms, while maintaining local management of data and products relevant to regional decision-making needs.
With a vision toward a global set of regional data cubes, it will be possible to take advantage of consistent time series satellite data and different, yet interoperable, datasets. Such data cubes and their corresponding open source application algorithms can be enhanced and shared across the world to address national and transboundary issues while maintaining data sovereignty and political separation through a regional implementation. In addition, to realise the full potential of the ODC products to address local and regional decision-making and policies, it is important to increase research and gather in-situ ground data for proper algorithm and product validation. Over time, it is expected that open data products will increase, their accuracy will improve, and data access and use will become easier and faster for everyone.