Configuring Submission Node for Torque 2.5


If you are planning to have more nodes where the users can do submission apart from the Head Node of the Cluster, you may want to configure a Submission Node. By default, TORQUE only allow one submission node. There are 2 ways to configure this submission node. One way is by using the Using RCmd authentication, the other is by using the “submit_host paramter” in the Torque Server

Step 1a: Configuring the Submission

First and Foremost, one of the main prerequistics is that the submission nodes must be part of the resource pool identified by the Torque Server. If  you are not part of the Torque Server, you may want to follow the steps to make the to-be-submission node part of the resource pool or a pbs_mom client. You can check the setup by looking at the Installing Torque 2.5 on CentOS 6 with xCAT tool, especially B. Configuring the TORQUE Clients. You might want to follow up with this optional setup Adding and Specifying Compute Resources at Torque to make sure your cores count are correct

Step 1b: Ensure the exchange keys between submission node and Torque Server

For more information, see Auto SSH Login without Password

Step 1c: Configure the submission node as a non-default queue (Optional)

For more information, see Using Torque to set up a Queue to direct users to a subset of resources

Step 2: Registering the Submission Node in Torque

If you do not wish the compute node to be a compute resource, you can put a non-default queue or unique queue which users  will  not send to.

Once you have configured the to-be-submission node as one of the client, you have to now to configure the torque server by this commands

# qmgr -c 'set server submit_hosts = hostname1'
# qmgr -c 'set server allow_node_submit = True'

Step 3: Putting Submission Node inside Torque Server /etc/host.equiv

# vim /etc/hosts.equiv
submission_node.cluster.com

Step 4: Test the configuration

Do a

$ qsub -I nodes=1:ppn=8

You should see from the torque server that the job has been submitted via the submission node by doing a qstat -an

$ qstat -an

Step 5: Mount Maui Information from PBS/MAUI Server

From the MAUI Server, do a NFS, mount the configuration and binaries of MAUI

Edit /etc/exports

/opt/maui               Submission-Node1(rw,no_root_squash,async,no_subtree_check) 
/usr/local/maui         Submission-Node1(rw,no_root_squash,async,no_subtree_check)

At the MAUI Server, restart NFS Services

# service restart nfs

At the submission node, make sure you have the mount point /opt/maui and /usr/local/maui for the

At /etc/fstab, mount the file system and restart netfs

head-node1:/usr/local/maui    /usr/local/maui        nfs      defaults  0 0
head-node1:/opt/maui          /opt/maui              nfs      defaults  0 0
#service netfs restart

Resources:

  1. Torque Server document 1.3.2 Server configuration
  2. Unable to Submit via Torque Submission Node – Socket_Connect Error for Torque 4.2.7
  3. Bad UID for job execution MSG=ruserok failed validating user1 from ServerNode while configuring Submission Node in Torque
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s