Skip to Main content Skip to Navigation
Conference papers

Docker & HEP: containerization of applications for development, distribution and preservation

Abstract : HEP software stacks are not shallow. Indeed, HEP experiments' software are usually many applications in one (reconstruction, simulation, analysis, ...) and thus require many libraries - developed in-house or by third parties - to be properly compiled and installed. Moreover, because of resource constraints, experiments' software is usually installed, tested, validated and deployed on a very narrow set of platforms, architectures, toolchains and operating systems. As a consequence, bootstrapping a software environment on a developer machine or deploying the software on production or user machines is usually perceived as tedious and iterative work, especially when one wants the native performances of bare metal. Docker containers provide an interesting avenue for packaging applications and development environment, relying on the Linux kernel capabilities for process isolation, adding "git"-like capabilities to the filesystem layer and providing (close to) native CPU, memory and I/O performances. This paper will introduce in more details the modus operandi of Docker containers and then focus on the hepsw/docks containers which provide containerized software stacks for -among others- LHCb. The paper will then discuss various strategies experimented with to package software (e.g. CVMFS, RPMs, from-source) and applied to optimize provisioning speed and disk usage, leveraging the caching system of Docker. Finally, the paper will report on benchmarks comparing workloads on native, VM and Docker containers setups.
Complete list of metadatas

http://hal.in2p3.fr/in2p3-01141690
Contributor : Sabine Starita <>
Submitted on : Monday, April 13, 2015 - 3:19:21 PM
Last modification on : Wednesday, September 16, 2020 - 5:45:22 PM

Identifiers

  • HAL Id : in2p3-01141690, version 1

Collections

Citation

S. Binet, B. Couturier. Docker & HEP: containerization of applications for development, distribution and preservation . Conference on Computing in High Energy and Nuclear Physics – CHEP2015, Apr 2015, Okinawa, Japan. ⟨in2p3-01141690⟩

Share

Metrics

Record views

142