I red the following trick in the doc : https://baobab.unige.ch/enduser/src/enduser/submit.html?highlight=sacct#multithreaded-jobs
I would like to pass to my python script the number of cores that must be used for multithreading. How can I proceed if I use the script above, which is designed to automatically assign all cores on a given node to the job being launched ?
As an example, what I currently do is that I set manually the --cpu_per_task option to 8, 12, 16… and then pass to my python script the environment variable $SLURM_CPUS_PER_TASK, which is automatically set to the --cpu_per_task value.
But in the case where this value is set by the --cpu_bind=mask_cpu:0xffffffff option, how can I know the effective number of cores being used, at execution time ?
I’m thinking of using the
multiprocessing.cpu_count() method of the
multiprocessing module of python, but I red somewhere that this may not be the right choice on a cluster, because it sometimes does not reflect the ‘real’ number of cores available.
How would you do, please ? Thanks in advance