Meetings > PI / Science Team Meeting > Posters

ARM Data Flow Revolution

 

Authors

Cory Stuart — Oak Ridge National Laboratory
Sherman Beus — Pacific Northwest National Laboratory
Robert James Records — Oak Ridge National Laboratory
Anthony Clodfelter — Oak Ridge National Laboratory
Laura Michelle Davies — Oak Ridge National Laboratory

Category

ARM infrastructure

Description

Currently, ARM data is collected on a timed basis. There are scheduled tasks that run to pick up data from systems, then forward it to the ARM Data Center (ADC). Within the ADC, there are scheduled tasks to move the data through the different phases of processing and distribution. There are many disparate tools to monitor and manage these processes. It is desirable to have this entire process be event driven, such that as soon as data is generated on an instrument, it is pushed through the collection, transfer, and processing phases automatically such that it is available to users within an hour or two. This poster will outline the current path and tools used, as well as the projected tools and methods that will be used in the next year.