You are here: TWiki> Online Web>RawData (2008-11-05, NikoNeufeld)
Tags:
create new tag
, view all tags

Writing files using the runcontrol

For this you need to be under dynamic allocation. Run-numbers will be allocated automatically from the Rundatabase. Your files are stored locallly under

/daqarea/lhcb/data/2008/RAW
This is mounted on the plus nodes. We do not want to mount it on other systems to ease the load on our NFS servers. These files are regularly (every hour) copied to CASTOR. You will find them there under
/castor/cern.ch/grid/lhcb/data/2008/RAW
From RAW onward the structure is the same
PARTITION/ACTIVITY/RUN

Pathname conversion convention

In a valid path to a data-file on the online system, substitute the

/daqarea 
by
/castor/cern.ch/grid
to obtain a valid path on CASTOR. The common file name part is also the LFN (Logical File Name in the GRID namespace).

Example

Run number 21444 for RICH1 was taken in "ALICE" activity. On plus02 you find it on

/daqarea/lhcb/data/2008/RAW/RICH1/ALICE/21444/021444_0000002461.raw

and on CASTOR

/castor/cern.ch/grid/lhcb/data/2008/RAW/RICH1/ALICE/21444/021444_0000002461.raw

Notes

  • If you don't find a file on CASTOR, don't panic. Files are only copied when they are not of zero-length and at least 10^6 bytes long.
  • Files are not automatically deleted from the /daqarea. So nothing will vanish, before you will get an explicit warning
  • Files are never deleted from CASTOR, unless you explicitly ask for this
  • Check the Run Database, if you think the file should be on CASTOR or in the Bookkeeping but it is not then send a mail to the helpdesk.

Writing files locally (using DiskWR)

PLEASE NOTE THAT THIS IS DEPRECATED - YOU CAN NOT WRITE FILES ANYMORE IN THIS WAY

When writing files locally, that is by using DiskWR, the files are not managed centrally by the Run Database. These files were regulary (every hour) copied to CASTOR. You will find them there under

/castor/cern.ch/lhcb/online/point8/<group>/

Notes

  • All special characters in your file names: blanks, \, ^, %, $, &, ? will be automatically converted to and "_": the handling/meaning of these characters is platform dependent on the platform and in additions is not supported by CASTOR. You can safely create use them to describe the run types, partition names.
  • small files (< 1 MB) will not be copied to CASTOR - pack them into archives
  • files which are marked as migrated in CASTOR (a small m in the nsls output) can and will eventually be deleted from the Online storage. We never delete files from CASTOR
  • Our scripts are paranoid - if after a file has been copied to CASTOR, you change the size of the file (rewrite, truncate, etc...) - the new file will not be copied and the original file on CASTOR will never be overwritten. Use a new file-name

-- NikoNeufeld - 05 Nov 2008

Topic revision: r4 - 2008-11-05 - NikoNeufeld
 

TWIKI.NET
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback