PBS/Torque

Funnel can be configured to submit workers to PBS/Torque by making calls to qsub.

The Funnel server needs to run on a submission node. Configure Funnel to use PBS by including the following config:

It is recommended to update the submit file template so that the funnel worker run command takes a config file as an argument (e.g. funnel worker run --config /opt/funnel_config.yml --taskID {{.TaskId}})

Compute: pbs

PBS:
    Template: |

#!/bin/bash #PBS -N {{.TaskId}} #PBS -o {{.WorkDir}}/funnel-stdout #PBS -e {{.WorkDir}}/funnel-stderr {{if ne .Cpus 0 -}} {{printf "#PBS -l nodes=1:ppn=%d" .Cpus}} {{- end}} {{if ne .RamGb 0.0 -}} {{printf "#PBS -l mem=%.0fgb" .RamGb}} {{- end}} {{if ne .DiskGb 0.0 -}} {{printf "#PBS -l file=%.0fgb" .DiskGb}} {{- end}} funnel worker run --taskID {{.TaskId}}

The following variables are available for use in the template:

VariableDescription
TaskIdfunnel task id
WorkDirfunnel working directory
Cpusrequested cpu cores
RamGbrequested ram
DiskGbrequested free disk space
Zonerequested zone (could be used for queue name)

See https://golang.org/pkg/text/template for information on creating templates.