The auto assigning for nodes is not smart enough

[yuema137]

Sometimes, the jobs are submitted to nodes that are already occupied by other jobs so the new jobs are always PD. But if we check the node status, for example:
nodestatus xenon1t
We will see that there are actually a lot of usable nodes. Then if the user specify the available node, the job can be submitted immediately.

So it seems that the auto assignment of nodes in slurm is not smart enough. But this cannot be changed by us as it's set by UChicago cluster. In order to solve this problem I think we should add a layer in batchq to allow users to scan for available nodes and only include them for submission.