Advancements in the fields of experimental particle, astroparticle, hadron, and nuclear physics have produced ever-increasing amounts of research data. Higher resolution of modern instruments have led to huge data sets, reaching the order of millions of terabytes and beyond, and require not only extensive computational and storage resources but also their user-friendly access.
FIDIUM aims to solve these challenges by virtually combining several complex infrastructures that offer a number of services. This includes pooling together dedicated CPU and storage resources at large data centers, temporary resources at HPC centers of partners, and commercial cloud systems. Providing an production system that can handle this highly heterogenous environment requires a high degree of abstraction and the development of the necessary software tools.
For processing the corresponding data sets, FIDIUM will combine modern storage systems of several data centers into data lakes. For unified and efficient data lake access, FIDIUM will use temporary caches at the local sites so that HPC resources can be incorporated into scientific workflows, e.g., from high-energy physics. Hierarchical storage management and intelligent data placement will utilize ad hoc file systems, among others.
Funding
FIDIUM is funded by the Germany Federal Ministry of Education and Research (BMBF) from 01.10.2021 — 30.09.2024.
Project Partners
- Rheinisch-Westfälische Technische Hochschule Aachen, Coordinator
- Rheinische Friedrich-Wilhelms-Universität Bonn
- Goethe Universität Frankfurt am Main
- Albert-Ludwigs-Universität Freiburg
- Georg-August-Universität Göttingen
- Universität Hamburg
- Karlsruher Institut für Technologie
- Ludwig-Maximilians-Universität München
- Bergische Universität Wuppertal
Associated Partners
- CERN
- DESY
- GridKa
- GSI Helmholtzzentrum für Schwerionenforschung