If you are planning to have more nodes where the users can do submission apart from the Head Node of the Cluster, you may want to configure a Submission Node. By default, TORQUE only allow one submission node. There are 2 ways to configure this submission node. One way is using the “submit_hosts paramter” in the Torque Server.
Step 1a: Configuring the Submission
First and Foremost, one of the main prerequisites is that the submission nodes must be part of the resource pool identified by the Torque Server. If you are not part of the Torque Server, you may want to follow the steps to make the to-be-submission node part of the resource pool or a pbs_mom client. You can check the setup by looking at the Installing Torque 4.2.5 on CentOS 6. Configuring the TORQUE Clients. You might want to follow up with this optional setup Adding and Specifying Compute Resources at Torque to make sure your cores count are correct.
Step 1b: Ensure the exchange keys between submission node and Torque Server
For more information, see Auto SSH Login without Password
Step 1c: Configure the submission node as a non-default queue (Optional)
For more information, see Using Torque to set up a Queue to direct users to a subset of resources
Step 2: Registering the Submission Node in Torque
If you do not wish the compute node to be a compute resource, you can put a non-default queue or unique queue which users will not send to.
Once you have configured the to-be-submission node as one of the client, you have to now to configure the torque server by this commands.
# qmgr -c 'set server submit_hosts = hostname1'
# qmgr -c 'set server allow_node_submit = True'
Step 3: Putting Submission Node inside Torque Server /etc/hosts.equiv
# vim /etc/hosts.equiv submission_node.cluster.com
Step 4a: Copy trqauthd from primary submission node to the secondary submission node
# scp -v /etc/init.d/trqauthd root@submission_node.cluster.com:/etc/init.d
Step 4b: Start the trqauthd service on the submission node
# service trqauthd start
Step 5: Test the configuration
$ qsub -I nodes=1:ppn=8
You should see from the torque server that the job has been submitted via the submission node by doing a qstat -an
$ qstat -an
Step 6: Mount Maui Information from PBS/MAUI Server
From the MAUI Server, do a NFS, mount the configuration and binaries of MAUI
/opt/maui Submission-Node1(rw,no_root_squash,async,no_subtree_check) /usr/local/maui Submission-Node1(rw,no_root_squash,async,no_subtree_check)
At the MAUI Server, restart NFS Services
# service restart nfs
At the submission node, make sure you have the mount point /opt/maui and /usr/local/maui for the
At /etc/fstab, mount the file system and restart netfs
head-node1:/usr/local/maui /usr/local/maui nfs defaults 0 0 head-node1:/opt/maui /opt/maui nfs defaults 0 0
#service netfs restart
- Torque Server document torqueAdminGuide-4.2.7
- Bad UID for job execution MSG=ruserok failed validating user1 from ServerNode while configuring Submission Node in Torque
- Unable to Submit via Torque Submission Node – Socket_Connect Error for Torque 4.2.7