Skip to content

Commit

Permalink
Update batch_job.md
Browse files Browse the repository at this point in the history
  • Loading branch information
ktiits authored Sep 5, 2024
1 parent d8c8f33 commit a32d635
Showing 1 changed file with 6 additions and 9 deletions.
15 changes: 6 additions & 9 deletions materials/batch_job.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,17 @@ When we submit a batch job script, the job is not started directly, but is sent
:::{admonition} How many resources to request?
:class: seealso

* You can use your workstation / laptop as a base measuring stick: If the code runs on your machine, as a first guess you can reserve the same amount of CPUs & memory as your machine has.
* If you have run the code runs on some other machine (your laptop?), as a first guess you can reserve the same amount of CPUs and memory as your machine has.
* You can also check more closely what resources are used with `top` on Mac and Linux or `task manager` on Windows when running on your machine.
* If your program does the same or similar thing more than once, you can estimate that the total run time is number of steps times time taken by each step.
* The first resource reservation is often a guess, that can be later adjusted.
* Before reserving multiple CPUs, check if your code can make use them.
* Before reserving multiple nodes, check if your code can make use them. Most GIS tools can not.
* You can also check more closely what resources are used with `top` on Mac and Linux or `task manager` on Windows when running on your machine
* Similarly for running time: if you have run it on your machine, you should reserve similar time in the supercomputer.
* If your program does the same thing more than once, you can estimate that the `total run time is number of steps times time taken by each step`.
* Likewise, if your program runs multiple parameters, the `total time needed is number of parameters times the time needed to run the program with one/some parameters`.
* You can also run a smaller version of the problem and try to estimate how the program will scale when you make the problem bigger.
* You should always monitor jobs to find out what were the actual resources you requested.
* When you double the number of cores, the job should run at least 1.5x faster.
* Some tools run both on CPU and GPU, if unsure which to use, a good rule of thumb is to compare the billing unit (BU) usage and select the one using less. A GPU uses 60 times more billing units than a single CPU core.
* You should always monitor jobs to find out what were the actual resources you requested.


Adapted from [Aalto Scientific Computing](https://scicomp.aalto.fi/triton/usage/program-size/)
Partly adapted from [Aalto Scientific Computing](https://scicomp.aalto.fi/triton/usage/program-size/)
:::

## Partitions
Expand Down

0 comments on commit a32d635

Please sign in to comment.