The ATLAS Eventindex: an event catalogue for experiments collecting large amounts of data

作者：D. Barberis, J. Cranshaw, Gancho Dimitrov, A. Favareto, Álvaro Fernández Casaní, S. González de la Hoz, J. Hřivnáč, D. Malon, Marcin Nowak, José Salt Cairols, J. Sánchez, R Sorokoletov, Qi Zhang · 发表于：Journal of Physics Conference Series · 年份：2014 · DOI：10.1088/1742-6596/513/4/042002 · 被引用次数：20 · 研究领域：Advanced Data Storage Technologies、Distributed and Parallel Computing Systems、Scientific Computing and Data Management

Modern scientific experiments collect vast amounts of data that must be catalogued to meet multiple use cases and search criteria. In particular, high-energy physics experiments currently in operation produce several billion events per year. A database with the references to the files including each event in every stage of processing is necessary in order to retrieve the selected events from data storage systems. The ATLAS EventIndex project is studying the best way to store the necessary information using modern data storage technologies (Hadoop, HBase etc.) that allow saving in memory key-value pairs and select the best tools to support this application from the point of view of performance, robustness and ease of use. This paper describes the initial design and performance tests and the project evolution towards deployment and operation during 2014.