Documentation for using the archiving scripts in this directory
---------------------------------------------------------------

Archiving of model output should be handled as part of model run 
post-processing.  This takes place in two parts: short-term archiving 
and long-term archiving.  These scripts are described below.

To date, these scripts have only been tested with stand-alone CAM and 
are not yet recommended for use elsewhere.  Eventually we'd like for 
these scripts to be used for CCSM runs as well.  For the moment, the list 
of supported machines is short, but will be expected to grow with machines 
and archival systems inside and outside of NCAR.  The supported machines 
are Bangkok, Calgary, Bluevista, Blueice, Tempest, and Lightning. 


*************
st_archive.sh
*************
This is the short-term archive script.  Its primary goal is to move model 
output out of the run directory to a temporary directory (the short-term 
archive directory) where the long-term archiver can process it later.  
This clears the run directory for the output of subsequent iterations 
of a model run.  You'll see in the template run scripts (run-ibm.csh, 
run-lightning.csh etc) that it is called in-line, immediately after the 
model run.  Required arguments are the root of the short-term archive 
directory (STA_ROOT) and the archiving casename (ARCH_CASE), passed in 
as environment variables. 

    STA_SAVE_INTERIM_RFILES
	FALSE (default) will only archive restart files from the end of 
	the last model run - any interim restart files generated will 
	be deleted; these will be stored in tar files that contain all 
	the files required for a model restart (just stand-alone CAM for now);
	this file will be called something like: 
	<casename>.all.r.yyyy-mm-dd-sssss.tar

Any output messages will appear in the log of the run script.



*************
lt_archive.sh
*************
This is the long-term archive script.  This is where most of the post-
processing related to archiving takes place.  Although it appears to be 
called in-line just after the short-term archiver, it actually spawns a 
batch job by default, so as to not delay a potential model resubmit. 
It requires STA_ROOT and ARCH_CASE as does the short-term archiver, 
but also the root of the long-term archive, LTA_ROOT.  The following 
environment variables are available as options:

    LTA_INTEGRITY         
	NORMAL (default) validates by file size, HIGH validates by 
	retrieval and comparison
    LTA_ANN_TARRING
	TRUE (default) enables the bundling of monthly history files 
	(secondary history files excluded) into yearly tar files
    LTA_RETENTION
	365 (default) days the file will be retained in the long-term 
	archive
    LTA_COS
	"rel=ec" (default) requests an economy class of service, meaning 
	only one copy will be saved; note this will likely require some 
	modification when non-NCAR systems are supported
    LTA_WPWD
	specifies a write password (blank by default) the read password 
	is also blank by default and there is no option for one currently
    LTA_PROJ
	specifies a project number for storage charges (uses the default 
	assigned to each user by default)

Any output message of lt_archive.sh will appear in the log file of the run 
script itself, but messages from the spawned job will appear in its own 
log file, residing in the directory it was invoked from.  Also in that 
directory will be the generated script used for the spawned job, and it'll 
be named: lta_<casename>_<machine>.sh.  Once generated, it can be used to 
run the long-term archiver manually and interactively, if desired.

The spawned job will return with a non-zero return code if all files made it 
to the long-term archive without error, and local copies were deleted.  Note, 
that copying to the long-term archive is done synchronously, and verification 
is done immediately after the copy.  

Files remaining in the short-term archive directory after completion of the 
long-term archive script can be handled in the following ways.  Users can 
wait for a subsequent iteration of the run script, and the files will be 
processed with the next batch of output files.  Or, users can manually invoke 
the long-term archive script (described elsewhere in this document) which is 
recommended when no more model runs are required.  The exception to this 
comes with "partial" files.  These are tar files created during bundling 
of history files into yearly tar files when output is incomplete for a 
particular year.  Normally, they just wait in the short-term archive directory 
until the remaining files for that year arrive, but if the model run ends 
mid-year and no subsequent iterations are expected, these partial files 
will require manual processing for them to reach the long-term archive.





<<<<more documentation likely to follow
