wiki:LIMS2_job_submission

Troubleshooting LIMS2 Job Submission

Here are a few things to check to ensure the LIMS2 job submission is running smoothly.

Make sure your instance has the data directories set up properly. If they are you should see files in them when you submit a job:

zollarsd@ultrascan [ ~ ]$ cd /srv/www/htdocs/uslims2/uslims2_zollarsd
zollarsd@ultrascan [ /srv/www/htdocs/uslims2/uslims2_zollarsd ]$ ll
...
drwxrwxr-x 3 zollarsd apache    4096 Jun 12 17:31 data
...
zollarsd@ultrascan [ /srv/www/htdocs/uslims2/uslims2_zollarsd ]$ ls data
v9tl0fmr29u70npo9rf415h354
zollarsd@ultrascan [ /srv/www/htdocs/uslims2/uslims2_zollarsd ]$ cd user
zollarsd@ultrascan [ /srv/www/htdocs/uslims2/uslims2_zollarsd/user ]$ ll
...
drwxrwxr-x 2 zollarsd apache    4096 Jun 12 12:29 data
...
zollarsd@ultrascan [ /srv/www/htdocs/uslims2/uslims2_zollarsd/user ]$                  

Check running processes to see if there are orphans and delete them. For instance, if the analysis queue is empty but there are entries like this---notice also that the date is old:

# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
...
apache    2797     1  0 2011  ?        00:00:00 perl /usr/local/ultrascan2/etc/grid/us_gridpipe.pl
apache    2798  2797  0 2011  ?        00:00:00 sh -c nohup perl -I$ULTRASCAN/etc $ULTRASCAN/etc/grid/us_tigre_job.pl /
srv/www/htdocs/uslims2/uslims2_cauma/data/i4615bgu5m4te000s9rfmmj8s2/1339602435.gc 118374 demeler@biochem.uthscsa.edu 1
20613104715 /srv/www/htdocs/uslims2/uslims2_cauma/data/i4615bgu5m4te000s9rfmmj8s2 /srv/www/htdocs/uslims2/uslims2_cauma
/data/i4615bgu5m4te000s9rfmmj8s2/experiments120613104715.dat /srv/www/htdocs/uslims2/uslims2_cauma/data/i4615bgu5m4te00
0s9rfmmj8s2/solutes120613104715-0.dat alamo.biochemistry.uthscsa.edu > $ULTRASCAN/etc/grid/log/gc_tigre_120613104715.1 
2> $ULTRASCAN/etc/grid/log/gc_tigre_120613104715.2
apache    2799  2798  0 2011  ?        00:00:00 perl -I/usr/local/ultrascan2/etc /usr/local/ultrascan2/etc/grid/us_tigr
e_job.pl /srv/www/htdocs/uslims2/uslims2_cauma/data/i4615bgu5m4te000s9rfmmj8s2/1339602435.gc 118374 demeler@biochem.uth
scsa.edu 120613104715 /srv/www/htdocs/uslims2/uslims2_cauma/data/i4615bgu5m4te000s9rfmmj8s2 /srv/www/htdocs/uslims2/usl
ims2_cauma/data/i4615bgu5m4te000s9rfmmj8s2/experiments120613104715.dat /srv/www/htdocs/uslims2/uslims2_cauma/data/i4615
bgu5m4te000s9rfmmj8s2/solutes120613104715-0.dat alamo.biochemistry.uthscsa.edu
...
#

Alternatively, stop and restart the grid service:

# /etc/rc.d/init.d/us_gridcontrol stop
Stopping US GridControl Service ...
# touch /var/lock/subsys/local
# /etc/rc.d/init.d/us_gridcontrol stop
Starting US GridControl Service ...
#

Check that both us_gridpipe_status_server.pl and us_gridpipe.pl are running:

# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
...
apache    1964     1  5 Jun12 ?        01:00:14 perl /usr/local/ultrascan2/etc/grid/us_gridpipe_status_server.pl 787
apache    1974  1963  0 Jun12 ?        00:00:00 perl /usr/local/ultrascan2/etc/grid/us_gridpipe.pl
...
#                                                                      

Verify permissions on grid log directory:

# echo $ULTRASCAN
/usr/local/ultrascan2
# cd $ULTRASCAN/etc/grid/
# ll
...
drwxrwxrwx 2 root   root      4096 Jun 13 10:47 log
...
# cd log
# ll
...
-rw-rw-rw- 1 root   root           0 Jun 12 17:12 gridpipe_status.stderr
-rw-rw-rw- 1 root   root    47968581 Jun 13 12:13 gridpipe_status.stdout
...
-rw-rw-rw- 1 root   root      156805 Jun 13 12:12 us_gridpipe.stderr
-rw-rw-rw- 1 root   root      359872 Jun 13 12:12 us_gridpipe.stdout
...
#
Last modified 7 years ago Last modified on Jun 13, 2012 4:30:57 PM