Background jobs

Why should I run jobs in the background?

Detaching jobs from your log in session will ensure that they will continue running even after you have logged out. This technique is particularly useful for running jobs on compute servers as it allows you to log out of the compute server and, if necessary, shut down down your desktop PC. Furthermore jobs on a server that continue running after you have logged out will not be affected if for any reason your desktop PC has a problem.

What about my input and output?

In many cases when packages such as R and Matlab are run, commands are typed in at the keyboard and the results are displayed on the screen. To create jobs that can safely be run without the need to remain logged into a server, we need to change this.

A simple R example

Let's looks at very simple example in R. In a data file called sp10.txt we have 10 days values for Standard & Poor's 500 index.

 Date         SP500
 3-Jan-95     459.11
 4-Jan-95     460.71
 5-Jan-95     460.34
 6-Jan-95     460.68
 9-Jan-95     460.83
10-Jan-95     461.68
11-Jan-95     461.66
12-Jan-95     461.64
13-Jan-95     465.97

We will read these values into an array and print them out. So we start R and enter the commands

dv <- read.table("SP10.txt", header=1)
sp10value = dv[,2]
sp500value

and see the output

 [1] 459.11 460.71 460.34 460.68 460.83 461.68 461.66 461.64 465.97 466.33

To run the job in the background the commands must be read from a file rather than typed in at the keyboard, and the output should be saved in a file rather than displayed on the screen.

Using an editor of your choice either type in or copy and paste your R commands into a file. It is conventional to give the file a .R suffix. In this case we could use a file called spjob.R which would contain the lines

dv <- read.table("SP10.txt", header=1)
sp10value = dv[,2]
sp500value

Now we can run the R job with

R CMD BATCH spjob.R &

and when the job has finished the file spjob.Rout will contain all the output from the R job as follows:

R version 2.9.0 (2009-04-17)
Copyright (C) 2009 The R Foundation for Statistical Computing
ISBN 3-900051-07-0

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

   Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

[Previously saved workspace restored]

> dv <- read.table("SP10.txt", header=1)

> sp500value = dv[,2]
> sp500value
 [1] 459.11 460.71 460.34 460.68 460.83 461.68 461.66 461.64 465.97 466.33
>
> proc.time()
   user  system elapsed
  0.633   0.046   0.704

The output is slightly confusing as our results are mixed up with informative R messages. An alternative approach is to change the commands so that the data itself is output to a separate file

dv <- read.table("SP10.txt", header=1)
sp10value = dv[,2]

sink(file="spjobdata.Rout")
sp500value 
sink(file="/dev/null")

This means that our data will now be written to spjobdata.Rout.

Help! Where will my pictures go? 

Many R sessions include the creation of plots. For example, a simple histogram could be created with the above data using the command

plot(sp10value, type="l")

If this command is included in spjob.R, the plot by default is saved in the same directory or folder, called Rplots.pdf.