QuerySpaces on Hadoop for the ATLAS EventIndex

Abstract : The new ATLAS EventIndex catalogue uses a Hadoop cluster to store information on each event processed by ATLAS. Several tools belonging to the Hadoop eco-system are used to organise the data in HDFS, catalogue it internally, and provide the search functionality. This presentation will describe the Hadoop-based implementation of the adaptive query engine serving as the back-end for the ATLAS EventIndex. The QuerySpaces implementation handles both original data and search results providing fast and efficient mechanisms for new user queries using already accumulated knowledge for optimisation. Detailed description and statistics about user requests are collected in HBase tables and HDFS files. Requests are associated to their results and a graph of relations between them is created to be used to find the most efficient way of providing answers to new requests. The environment is completely transparent to users and is accessible over several command-line interfaces, a Web Service and a programming API.
Type de document :
Communication dans un congrès
21st International Conference on Computing in High Energy and Nuclear Physics – CHEP2015, Apr 2015, Okinawa, Japan
Liste complète des métadonnées

http://hal.in2p3.fr/in2p3-01176580
Contributeur : Sabine Starita <>
Soumis le : mercredi 15 juillet 2015 - 15:46:06
Dernière modification le : jeudi 11 janvier 2018 - 06:26:23

Identifiants

  • HAL Id : in2p3-01176580, version 1

Collections

Citation

J. Hrivnac, R. Yuan, J. Cranshaw, A. Favareto, F. Prokoshin, et al.. QuerySpaces on Hadoop for the ATLAS EventIndex. 21st International Conference on Computing in High Energy and Nuclear Physics – CHEP2015, Apr 2015, Okinawa, Japan. 〈in2p3-01176580〉

Partager

Métriques

Consultations de la notice

86