"BigStorage: Storage-based Convergence between HPC and Cloud to address Big Data Challenges" is a European Training Network (ETN) to train future data scientists to enable them and us to apply holistic and interdisciplinary approaches for taking advantage of a data-overwhelmed world, which requires HPC and Cloud infrastructures with a redefinition of storage architectures underpinning them - focusing on meeting highly ambitious performance and energy usage objectives.
The drivers for this data deluge are twofold: the interest of enterprises and agencies in collecting, processing, and publishing heterogeneous data, derived from multiple sources as well as citizens publishing content through channels such as social networks and Cloud systems. To gain value from this data it must be analysed and often combined or compared with simulated and predicted data. This huge data collection, which cannot be managed by current data management systems, is known as Big Data. Techniques to address it are gradually combining with what has been traditionally known as High Performance Computing. Therefore, this ETN will focus on the convergence of Big Data, HPC, and Cloud data storage, ist management and analysis.
BigStorage is a European Training Network, project ID: 642963, funded by the EU H2020, Marie Sklodowska-Curie Actions
Project Partners
- Universidad Politécnica de Madrid (Coordinator)
- Barcelona Supercomputing Center
- Johannes Gutenberg Universität Mainz
- Foundation for Research and Technology – Hellas
- Inria
- Seagate
- Deutsches Klimarechenzentrum
- CA Technologies
- Fujitsu Technology Solutions GmbH
- Commissariat à l’Energie Atomique et aux Energies Alternatives (CEA)
Associated Partners
- IBM Research Ireland
- Bull SAS
- Informática El Corte Inglés
Funding Period
01/2015 -- 12/2018
Contact
2020
- Marc-André Vef, Nafiseh Moti, Tim Süß, Markus Tacke, Tommaso Tocci, Ramon Nou, Alberto Miranda, Toni Cortes, and André Brinkmann. 2020. GekkoFS - A Temporary Burst Buffer File System for HPC Applications. Journal of Computer Science and Technology (JCST) 35, 1: 72–91. DOI Author/Publisher URL
2019
- Danilo Oliveira, André Brinkmann, Nelson Rosa, and Paulo Romero Martins Maciel. 2019. Performability Evaluation and Optimization of Workflow Applications in Cloud Environments. Journal of Grid Computing 17: 749–770. DOI
- Danilo Oliveira, Jamilson Dantas, Nelson Rosa, Paulo R M Maciel, Rubens de S Matos, and André Brinkmann. 2019. A dependability and cost optimisation method for private cloud infrastructures. International Journal of Web and Grid Services (IJWGS) 15: 367–393. DOI
2018
- Marc-André Vef, Nafiseh Moti, Tim Süß, Tommaso Tocci, Ramon Nou, Alberto Miranda, Toni Cortes, and André Brinkmann. 2018. GekkoFS - A Temporary Distributed File System for HPC Applications. In IEEE International Conference on Cluster Computing (CLUSTER), Belfast, UK, September 10-13, 319–324. DOI Author/Publisher URL
2017
- Danilo Oliveira, Rubens de S Matos, Jamilson Dantas, Jo ao Ferreira, Bruno Silva, Gustavo Callou, Paulo R M Maciel, and André Brinkmann. 2017. Advanced Stochastic Petri Net Modeling with the Mercury Scripting Language. In Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS), 192–197. DOI
2021
- Nafiseh Moti, Frederic Schimmelpfennig, Reza Salkhordeh, David Klopp, Toni Cortes, Ulrich Rückert, and André Brinkmann. Simurgh: A Fully Decentralized and Secure NVMM User SpaceFile System. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC). Author/Publisher URL