Use of Compute Servers

Please take care to always use the computing facilities responsibly and to comply with these simple rules, otherwise your access will be withdrawn.

Users should

Use the compute server most appropriate to the size of the problem being solved, e.g. smaller jobs may comfortably run on blackbird, whereas larger ones need blackgull or blackgoose. This is to ensure the best use of facilities, whilst avoiding a single user using everything.
Choosing a server

Check how busy a server is before starting compute intensive jobs. If your job(s) will take any remaining resource above 90%, please wait or select another server if available.
Checking for free resources

Use no more than a third of any resource on a server, e.g. up to a third of the CPU cores, threads, memory, and disk space. This is per user not per job. You are responsible for your jobs and the resources they consume, ensuring resources are still available to others.
Monitoring with htop
Killing jobs

Email ithelp@stats.ox.ac.uk if you intend to run jobs lasting more than 24 hours. This helps us differentiate your jobs from potential runaways.

Running jobs in the background is good practice and allows you to run jobs after you've logged out. See here for more details. Finally, please start jobs with a low priority.

Disk Space

The disk space on these compute servers (/data/server_name) is intended to support active jobs and is not intended for medium or long term storage.

On request (please email ithelp@stats.ox.ac.uk) we normally create two directories for your use e.g.

          /data/server_name/a_user
and
          /data/server_name/not-backed-up/a_user

The first is backed up to HFS but the total changes for the whole server must not exceed 200 GB a day for everybody, otherwise OUCS/ITS aborts the backup. This is the ideal place to store your results while your jobs are running.

The backup size restriction does not apply to the not-backed-up areas, which are ideal for data which can be regenerated again or very large files and datasets.

/tmp space, for performance reasons, is now using memory rather than actual physical disk space, so it is much faster.

Swap space is no longer provided on the larger memory systems due to the significant drop in performance when systems are forced to swap to and from disk.
Monitoring disk usage

Booking a server

Users can book blackbird to run compute intensive jobs requiring more than a third of any resource. blackbird will be reserved for your dedicated use on a specific day, for up to a week, but if found idle will be released for others to use.

Please email ithelp@stats.ox.ac.uk if you wish to book blackbird, giving as much detail as possible including disk space requirements, when you will be ready to start, and duration of jobs so we can schedule bookings.

Exceptional circumstances

If your jobs do not easily fit within this structure please email ithelp@stats.ox.ac.uk and we can discuss the best solution for you. This may involve the use of other servers, virtual compute resources or the use of the Oxford Super Computing Centre.