Parallelization and loop for with R

Hello,

I suggest that you don’t use detectCores as it will detect the whole cores of the compute node, even if you are restricted a subset of them by Slurm and you’ll get poor performances.

Please have a look at the example we provide here.

If you are happy with the execution time of one instance, the best that you can do is to use Slurm job array to launch 50 instances of your job.

Be sure to do some benchmark first: launch your instance with 2, 4, 8, 20 cores and measure time. If the performance aren’t better with 4, stick with 2 cores, etc.