eXtreme DataCloud: Providing Scalable Distributed Storage for the European Open Science Coud

The core activity within the newly created H2020 project “eXtreme DataCloud” is the policy-driven orchestrating of federated data management for data intensive sciences like High Energy Physics, Astronomy, Photon and Life Science. Well-known experts in the field are working on combining already established data management and orchestration tools to provide a highly scalable solution supporting the entire European Scientific Landscape. The work will cover “data Life Cycle Management” as well as smart data placement on meta data, including storage availability, network bandwidth and data access patterns. Mechanisms will be put in place to trigger computational resources based on data ingestion and data movements. The quality of the requested or utilized storage is negotiated by an extension of the SNIA CDMI protocol. This presentation will elaborate on the architecture of this endeavor and on a new CDMI reference implementation, provided by INDIGO-DataCloud, the predecessor of the “eXtreme DataCloud”.

Learning Objectives:
1. Distributed storage management in the European Open Science Cloud
2. Distributed data flow orchestration for High Energy Physics and Photon Science
3. Automatic Data Processing in the Cloud on high performance data ingest