EOhopS: EO data hosted processing for science

Within the Third Party Mission (TPM) scheme, ESA provides to Earth Observation users data from non-ESA missions, so-called Third Party Missions, to complement the data from ESA EO missions and to support and build up scientific user community for those data in Europe.

In order to gain access to the TPM data, ESA establishes co-operation schemes with the owners/operators of the TPM and/or procures data provisioning services.

Currently, TPM data is provided to users physically by download from ESA archives or from data providers’ servers. In the future, ESA intends to offer users also the possibility to process TPM data in hosted processing infrastructures. In that case, users will not download the original EO data, but only view and process it, and download results of their processing. In such a model, a number of challenges need to be tackled, as licence conditions of EO Data-as-a-Service, allocation of processing capabilities to users, connections of the processing infrastructure to TPM data providers for ordering and archive access, etc.

The purpose of this project is to gain experience with regard to those challenges and to test user acceptance of such a model. Scope of the project is to provide to scientific users access to EO data in a hosted processing infrastructure allowing them to process the data and download the results. ESA intends to issue an announcement of opportunity for this service. Upon evaluation of the proposals, ESA will allocate users certain budget to access and process EO data in the hosted processing infrastructure. ESA will then redirect users to a web-portal dedicated to this project, operated by CloudEO for the use of hosted processing service and consumption of the allocated budget.

Visit EOhopS Store

The Science Data Cloud combines a technology framework and a public cloud platform to provide Data-as-a-Service and Info-as-a-Service. This enables a multi-layered ecosystem of services to evolve that leverages the value from EO data provided by the European Space Agency.

The service is built on three stack layers, the front-end layer, orchestration layer and infrastructure layer. The infrastructure layer consists of the cloud orchestration, provisioning platform and core infrastructure, with data sets hosted in CloudSigma’s custom object storage solution. Upon this the Science Data PaaS layer is integrated to provide ontological data access and enable end-users to discover the service either via the cloud service marketplace or via an external third- party store. End-users also have the ability to discover applications, connect directly to the relevant data sets, and provision appropriately sized virtual servers accordingly.

The framework is tested against a Flood Delineation for Damage Assessment use-case, demonstrating how end-users - in this case regional civil protections agencies across Europe - can access datasets provided by ESA (including Sentinel-1) and process such data via the chosen application partner. Our vision is one where everyone can access data sets in an environment that allows immediate development and manipulation of the data and a ready onward market for the results of that effort.


The project OPUS-GMES creates an operational platform for provision and processing of Sentinel-data in support of Copernicus geoinformation services (BStMWi) (2013-2017) and is lead by DLR.

CloudEO platform acts as an example of a third party cloud platform where processes developed by research or industry partners can be offered to industry with secured intellectual property. Innovative and flexible licence and business models are supported for project work, development and production.


About cloudeo

cloudeo operates a unique, vendor-independent, data-agnostic market platform through which customers can obtain professional geoinformation services from leading national and international providers at low cost. Services include dedicated solutions for many industries; such as agriculture, urban and landscape planning, logistics, telecommunications, and water management. Customers are provided with high-resolution imagery, 3D terrain models, thematic maps, and sensor data for a wide range of applications. The TimeCaster™ service in our cloudeo web application is available on our store.

Big Earth Datacube Analytics Made Easy

BigDataCube project started to develop flexible and scalable services for massive spatio-temporal Earth Observation data.

Project coordinator, Jacobs University Bremen, invited the joint research consortium, consisting of rasdaman GmbH (Bremen), CloudEO AG (Munich) and DLR (Deutsches Zentrum für Luft- und Raumfahrt). The 18-month project is funded by the German Federal Ministry for Economic Affairs and Energy, BMWi.
The project will demonstrate integration of the innovative Earth datacube paradigm – i.e., analysis-ready spatio-temporal raster data – into commercial Earth Observation (EO) services as well as public cloud infrastructures. To this end, the worldwide leading European Datacube technology (in database lingo: "Array Database"), rasdaman, will be installed on the German Copernicus hub, CODE-DE, as well as in the commercial hosted processing environment of CloudEO to exemplarily offer federated analytics services.

Complementing the batch-oriented Hadoop service already available on CODE-DE, rasdaman will offer important additional functionality, in particular an interactive paradigm of “any query, any time, on any size”, strictly based on open geo standards and federated with other data centres, in particular geo services offered on CloudEO. On this platform novel, specialized services can be established by third parties in a fast, flexible, and scalable manner. For this particular project commercial data provided by Intermap and PlanetObserver, will be combined with open source data.

For industry and research, this will mean enhanced access to value-adding services supporting the collaboration across disciplinary and geographical boundaries. The BigDataCube project is managed by Peter Baumann, Professor of Computer Science at Jacobs University who emphasises “open standards for datacube analytics are available today, and have been proven on multi-Petabyte services. Now it is time to convince data users how they can benefit from such services on a routinely basis".

Contact:

    Prof. Dr. Peter Baumann, Jacobs University Bremen


BigDataCube Project Team

The BigDataCube project team: (from left to right): Stefan Wiehle (DLR), Dr. Ursula Benz (CEO CloudEO AG), Dimitris Bellos (CloudEO AG), Simon Tzvetanov (CloudEO AG), Vlad Merticariu (Jacobs University Bremen, rasdaman GmbH), Sven Jacobsen (DLR), Brennan Bell (Jacobs University Bremen), Bang Pham Huu (Jacobs University Bremen), Dimitar Misev (Jacobs University Bremen, rasdaman GmbH), Prof. Dr. Peter Baumann (CEO rasdaman GmbH, Jacobs University Bremen). Photo: Heike Hoenig, rasdaman GmbH


BMWi_en

The project TEISS (Thales ESA INFOaaS Stimulus) is one of four projects of ESA to stimulate the development of information as a service. Data, service and cloud partners demonstrate together highly automated end-to-end workflows providing information services based on Earth Observation data. Within the project it is analysed how these workflows can use joint resources like Helix Nebula Marketplace to reach competitive pricing and deliver the information fast and easy to access to the customer.
Thales Services is leading this project. Project partners are CloudEO and Sixq. CloudEO Store acts as customer interface to order services or request quotes. Please find a first list of services from EOMAP and Geocledian here.

About cloudeo
cloudeo operates a unique, vendor-independent, data-agnostic market platform through which customers can obtain professional geoinformation services from leading national and international providers at low cost. Services include dedicated solutions for many industries; such as agriculture, urban and landscape planning, logistics, telecommunications, and water management. Customers are provided with high-resolution imagery, 3D terrain models, thematic maps, and sensor data for a wide range of applications. The TimeCaster™ service in our cloudeo web application is available on our store.

CIINDY - CloudEO GeoMarketplace and IPT Poland Integration Study

The European Space Agency (ESA) has funded the EO Innovation Platform Testbed Poland to give with this EO Cloud easy access to Sentinel and Landsat data to researchers and industry. CloudFerro is building up the infrastructure and offers since 2016 access for first users.
Within the project CIINDY studies are performed how a connection between the GeoMarketplace CloudEO and the EO Cloud in Poland can be beneficial for developers and providers of GeoServices in terms of better and faster access to the market and more cost efficient production.

CloudEO teamed up for the studies with Exelis VIS, a Harris Geospatial Solutions company and with Science [&] Technology from Norway.

CIINDY - I

Within CIINDY-I the first prototypical connection of CloudEO Store to the EO Cloud was set up to let customers order on CloudEO Store a change detection analysis for a certain area using Landsat or Sentinel data.
We are proud on the success of this project.

  • Now anyone who wants to order the GeoService change detection from Harris Geospatial Solutions can do so on the CloudEO GeoMarketplace and will benefit from the fast execution on the EO Cloud.
  • Much more, every remote sensing or GIS scientist developing ENVI or IDL analytics and wanting to take benefit from the connection CloudEO GeoMarketplace and EO Cloud can do so easily and without any extra costs for this powerful software package!

To achieve this goal, Exelis VIS and CloudEO teamed up. Together with the support of Cloud Ferro the workflow was set up and the first integration between the EO Cloud and CloudEO Store is operational.
The customer can select between a multitude of change detection parameters and select his desired scenes from the full data catalogue.
Once the customer orders the change detection, it is executed automatically on the EO cloud. This includes the access to the data.
If Sentinel-2 data are used or Landsat data on Europe, the data are on the platform and thus the EO cloud gives a big time advantage for the analysis. The process can be executed within few seconds or minutes. However, the process also executes without human intervention, if the data are not available, e.g. Landsat data outside Europe. In this case an additional process automatically request the scenes from Amazon Web Services and starts the change detection process as soon as the data are available.

CIINDY - II Onboarding of GeoService Silvisense

Within CIINDY II, the Norwegian company Science and Technology and CloudEO teamed up to onboard the GeoService Silvisense onto CloudEO GeoMarketplace and to explore together the benefits the EO Cloud on IPT Poland brings in terms of easy data access and processing capabilities.
CloudEO uses its experience from CIINDY I to connect the Store through CloudEO Order Orchestration with the EO Cloud of CloudFerro.
Already the basic version of Silvisense, a detailed land cover map, provides a helpful input for companies and institutions in the forestry, agriculture and land planning business. This map serves as entry to advanced forest health analytics from Silvisense or as input for other value adders to accelerate and improve their processes.
Main project tasks
CloudEO presents the GeoService on CloudEO Store and implements the e-Commerce system.
Customers will be able to order this GeoService as one time analysis or subscribe to it as a monitoring service. The pricing is determined by the size of the selected area, the geometric resolution of the analysis, the length of the monitoring period and the detail of the analysis.
Different payment options are possible. In any case direct bank transfer is enabled. For order volumes below 500 Euro payment with credit card or paypal can be selected. For order volumes larger than 3000 Euro, direct bank transfer and payment on invoice is offered. New customers need to pay in advance.
With ordering, the customer accepts the Terms and Conditions of CloudEO. These conditions cover not only the needs of CloudEO but also the needs of the GeoService provider and avoid the need for the customer to sign multiple Terms and Conditions to consume one GeoService.
Once the payment is received the CloudEO Order Orchestration system triggers and controls the processing chain. The customer will find this information in the tab "Current Services" in his dashboard. When the results are produced, the customer gets notified and receives a link where he can download the results.
Project progress:

  • With milestone 1, May 19th, anyone interested can already find the GeoService on CloudEO Store and order it.
  • With milestone 2, July 31st, the full order, payment, processing and delivery chain will be implemented, providing in a fully automated workflow the results within few hours after data availability.


Is it technically and economically feasible to use commercial remote sensing imagery for crowdsourcing to validate geo-data?

This question is one aspect within the ESA Project: “Thematic Exploitation Platform URBAN (TEP Urban)” which was addressed in a study, subcontracted from DLR to the companies Pallas Ludens and CloudEO.

To get first answers, Pallas Ludens and CloudEO, set up a prototype presenting multi-resolution imagery to the crowd to validate DLR’s Global Urban Footprint (GUF) layer over selected test areas. First results confirm the technical feasibility and with new pay per use business models and fine-grained usage control commercial medium, high and very high imagery can be considered.


About cloudeo

cloudeo operates a unique, vendor-independent, data-agnostic market platform through which customers can obtain professional geoinformation services from leading national and international providers at low cost. Services include dedicated solutions for many industries; such as agriculture, urban and landscape planning, logistics, telecommunications, and water management. Customers are provided with high-resolution imagery, 3D terrain models, thematic maps, and sensor data for a wide range of applications. The TimeCaster™ service in our cloudeo web application is available on our store.


Initiated at the 4th Ground Segment Coordination Body Workshop on 24th September 2015 at ESA-ESRIN (Frascati, Italy), the Copernicus World Alliance (CWA) is gathering 30+ European industry stakeholders addressing Earth Observation (EO) Services.

The Copernicus World Alliance is

  • helping the EO community to tackle the challenges coming from the digital economy
  • supporting EARSC initiative for creating a European MarketPlace for EO Services (dowload pdf here)
  • promoting INFOrmation as a Service (INFOaaS) based on Earth Observation data and services which bring value to public and private end-users able and willing to pay such services

For example, you may be interested in tiny but smart EO services such as the ones described below.