Automated Supervision of Data Production - Managing the Creation of Statistical Reports on Periodic Data

Data production systems are generally very large, distributed and complex systems used for creating advanced (mainly statistical) reports. Typically, data is gathered periodically and then subsequently aggregated and separated during numerous production steps. These production steps are arranged in a specific sequence (workflow or production chain), and can be located worldwide. Today, a need for improving and automating methods of supervision for data production systems has been recognized. Supervision in this context entails planning, monitoring and controlling data production. Two significant approaches are introduced here for improving this supervision. The first is a 'closely-coupledd' approach (meaning direct communication between production jobs and supervisory tool, informing the supervisory tod immediately about delays in production) - based upon traditional production planning methods typically used for manufacturing (goods) and adopted for working with data production. The second is a 'loosely-coupled' approach (meaning no direct communication between supervisory tool and production jobs is used) - having its origins in proven traditional project management. The supervisory tool just enquires continuously the progress of production. In both cases, dates, costs, resources, and system health information is made available to management. production operators and administrators to support a timely and smooth production of periodic data. Both approaches are theoretically described and compared. The main finding is that, both are useful, but in different cases. The main advantages of the closely coupled approach are the large production optimisation potential and a production overview in form of a job execution plan, whereas the loosely coupled method mainly supports unhindered job execution and offers a sophisticated production overview in form of a milestone schedule. Ideas for further research include investigation of other potential approaches and theoretical and practical comparison.