wiki:MWLNotes

This page is for MWL analysis. It is a work in progress.

Getting the source

The MWL code has not been integrated into the primary US II source code. Please contact e m r e at b i o c h e m dot u t h s c s a dot e d u if you wish to obtain the code.

Allocations

Since MWL analysis is so computationally intensive, you will need to arrange an allocation on some target cluster for your analyses.

The Setup

MWL code is currently command line driven and requires an account on bcf.uthscsa.edu. If you have another linux target where you wish to install the code, let us know and we will discuss the issues involved.

Directory Structure

Once setup, your account will contain a subdirectory 'mwl'. This is the base for all the mwl analysis.

Underneath mwl, there is a 'bin' directory which contains perl scripts to assist in the setup of analysis.

For each experiment, there will be a base directory under mwl.

Here is the structure of one of these, supposing the experiment is called 'EXP' and has one analysis labeled '1':

$HOME/mwl/EXP/ - base directory
$HOME/mwl/EXP/1 - 1st analysis of EXP data.
$HOME/mwl/EXP/1/exp*dat, sol*dat - these are the data files needed for analysis runs
$HOME/mwl/EXP/1/bbjob-?, bjob-? - these are the job submission scripts (particular to your allocation)
$HOME/mwl/EXP/1/*.index - one text file that contains the mapping from the wavelength index to the actual wavelength

these files are present after a successful analysis run:
$HOME/mwl/EXP/1/jobs/ - subdirectory containing analysis results
$HOME/mwl/EXP/1/jobs/mwl-* - subdirectories containing analysis of individual wavelengths
$HOME/mwl/EXP/1/rmsds - a manually created file that contains the rmsds of the analysis runs
$HOME/mwl/EXP/1/results/ - subdirectory that contains the model files of the results
$HOME/mwl/EXP/1/results/*-model.sa2d_mc_dis.1* - the distribution files created from the models
$HOME/mwl/EXP/1/results/combine.sa2d_mc_dis.11 - a combined distribution file (all wavelengths)
$HOME/mwl/EXP/1/models/ - subdirectory that contains the subdirectories for models for use with us_viewmwl
$HOME/mwl/EXP/1/models/model/ - subdirectory that contains the primary models
$HOME/mwl/EXP/1/models/w_noise-model/ - subdirectory that contains the models with ti/ri noise added in
$HOME/mwl/EXP/1/models/experiment_wo_noise-model/ - subdirectory that contains the models with ti/ri noise subtracted
$HOME/mwl/EXP/1/models/w_baseline-model/ - subdirectory that contains the models with baseline correction added

Performing a MWL analysis

Currently, only 2DSA analysis is supported, although GA analysis is possible

First, extract a few sample wavelengths of data and process it normally to obtain 2DSA run parameters that are applicable to the range of wavelengths you wish to process.

Make a subdirectory under $HOME/mwl for this experiment and place the binary data file there:
(e.g. EXP2) $ mkdir $HOME/mwl/EXP2 $HOME/mwl/EXP2/exps

$ cp $HOME/mwl/bin/makegc.pl $HOME/mwl/EXP2/

edit makegc.pl to set the appropriate directories and run parameters and expt-number (under the array @exps).
From $HOME/mwl/EXP2/: $ perl makegc.pl skip-scan-count - this will create the analysis files in a subdirectory $HOME/mwl/EXP2/expt-number/
There will be a file created $HOME/wml/EXP2/exps/p'expt-number'.tbz, this needs to be copied to the target analysis cluster

Extract p'expt-number'.tbz on the target cluster and run with target cluster specific instructions.
When the jobs have completed on the target cluster, recover the results. MORE INFO NEEDED.

Example on juropa

$ cd $HOME/mwl/EXP2/exps/
# ----- copy the job over:
$ scp p'expt-number'.tbz vip.rzg.mpg.de:        
$ ssh vip.rzg.mpg.de
# ----- working from vip.rzg.mpg.de, the file needs to be copied over to juropa
$ scp p'expt-number'.tbz juropa.fz-juelich.de:
$ ssh juropa.fz-juelich.de
# ----- now working from juropa, setup the job
$ mkdir -p $WORK/rzg0kpla/mwl/'expt-number'/jobs
$ cd $WORK/rzg0kpla/mwl/'expt-number'
$ tar jxf ~/p'expt-number'.tbz
# ------ submit them
$ for i in bbjob-{?,??}; { echo $i; msub $i; echo $i submitted; }
# ------ monitor the job:
$ showq -u rzg0kpla 
# ------ when the job(s) is(are) done, make sure all completed
$ cd $WORK/rzg0kpla/mwl/'expt-number'
# ------ this will give you the number of jobs that should have been completed
$ echo exp*dat | wc -w
# ------- compare with the number run
$ cd jobs; echo mwl-* | wc -w
# ------- if they don't match, something didn't run & you'll have to dig further
# ------- now make sure all the jobs completed successfully
$ cd ~; perl bin/checkcomplete.pl 'expt-number' > runlog
# ------- an empty runlog file will mean they were all ok, otherwise it will contain a list of failed jobs, which should be investigated further
$ cat runlog
# ------- package and copy back the results
$ cd $WORK/rzg0kpla/mwl/'expt-number'
$ tar jcf ~/r'expt-number'.tbz jobs/mwl-*
$ scp ~/r'expt-number'.tbz myuser@bcf.uthscsa.edu:mwl/EXP2/exps/r'expt-number'.tbz
$ exit
# ------- now on vip.rzg
$ exit
# ------- now on bcf
$ cd $HOME/mwl/EXP2/exps/'expt-number'
$ tar jxf r'expt-number'.tbz

Viewing Results with us_viewmwl

1st load the MWL binary data.
Then 'Load MWL model': one file from $HOME/mwl/EXP/1/models/*/*.dat and all wavelengths will be loaded (make take awhile).
Click 'show model' and then Update Plot (2D/Radius mode) and you will see superimposed results (in red) with your data.

Further processing

MORE INFO NEEDED.
combining the distributions.

Last modified 8 years ago Last modified on Aug 28, 2010 6:23:36 PM