Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slurm resource named clusters in docs, and cluster in getClusters #297

Open
jemus42 opened this issue Oct 17, 2023 · 0 comments
Open

Slurm resource named clusters in docs, and cluster in getClusters #297

jemus42 opened this issue Oct 17, 2023 · 0 comments

Comments

@jemus42
Copy link

jemus42 commented Oct 17, 2023

I can't quite decide which way it is.

In clusterFunctionsSlurm.R, the getClusters function is defined which looks for a resource named cluster (singular).

getClusters = function(reg) {
clusters = filterNull(lapply(reg$resources$resources, "[[", "cluster"))
if (length(clusters))
return(stri_flatten(unique(as.character(clusters)), ","))
return(character(0L))

The documentation of submitJobs refers to clusters (plural):

#' \item{clusters:}{Resource used for Slurm to select the set of clusters to run \code{sbatch}/\code{squeue}/\code{scancel} on.}

The problem:

if this resource / getClusters is NULL, there's no cluster specified and functions like findRunning() etc. will return an empty table despite jobs are running.
For months I was wondering why my queued Slurm jobs where listed as expired and I assumed I was doing something wrong or my template was outdated.
Took me a moment to figure out that I was accidentally correctly misspecifying clusters in my resources = list(...) call 🥴

EDIT: Oh, and the template I'm using relies on resources$clusters.

Suggested fix

I'm not sure. I was about to prepare a PR when I realized that I don't want to

  • change the internal getClusters function to look for clusters instead of cluster because that will silently break peoples existing batchtools configs and they might not notice.
  • change the docs for submitJobs to (wrongly?) suggest that the resource should be named cluster, implying that multiple clusters can not be specified (which I never tried)

At the very least I thought about introducing an assertion on provided resources to shield against this sort of thing, but I'm not sure how to go about that yet.

@jemus42 jemus42 changed the title Slurm resource name clusters misspecified either in docs or internally Slurm resource named clusters in docs, and cluster in getClusters Oct 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant