Problem with python library path

HPC Technical
1

Hi,
I am new to HPC. I am trying to run a demanding script on brain images in python and I run into the following error:

> error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file:

It seems to be a somewhat common error (see here for instance)
but I haven^t been able to apply the proposed solutions.

Has anyone run into this before? Strangely, the first time I ran the script (well it crashed for other reasons), I did not encounter this problem. Nor did my test scripts.

Thanks for your help.
Leyla

2

Hi,
Could you please copy paste your sbatch script?

3

Hi,
Here it is:

#!/bin/bash
#SBATCH --job-name=FMRC_Test    # Job name
#SBATCH --mail-type=END,FAIL          # Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=lkhenissi@gmail.com     # Where to send mail
#SBATCH --partition=public-cpu
#SBATCH --ntasks=4                    # Run on a single CPU
#SBATCH --cpus-per-task=1                    # cpu per task
#SBATCH --mem=1gb                     # Job memory request
#SBATCH --time=72:00:00               # Time limit hrs:min:sec
#SBATCH --output=serial_test_%j.log   # Standard output and error log pwd; hostname; date


. ~/baobab_python_env/bin/activate

echo "Running FREMC script on single CPU"
~/baobab_python_env/bin/python

/home/users/l/louedkhe/LSS_Surprise/DecodeYggdrasilFREMC.py

echo 'DONE'

Thank you.

4

Hi, you are missing module load XXX where XXX is a Python version before the line with activate

5

Thank you Yann, I was just about to say I fixed the issue as follows, based on the UCL cluster’s FAQ.
My bash script now looks like this:


#!/bin/bash
#SBATCH --job-name=FMRC_Test    # Job name
#SBATCH --mail-type=END,FAIL          # Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=lkhenissi@gmail.com     # Where to send mail
#SBATCH --partition=public-cpu
#SBATCH --ntasks=4                    # Run on a single CPU
#SBATCH --cpus-per-task=1                    # cpu per task
#SBATCH --mem=1gb                     # Job memory request
#SBATCH --time=00:10:00               # Time limit hrs:min:sec
#SBATCH --output=serial_test_%j.log   # Standard output and error log pwd; hostname; date

module load  GCCcore/8.3.0
module load  Python/3.7.4
. ~/baobab_python_env/bin/activate
echo "Running FREMC script on single CPU"
cd LSS_Surprise

~/baobab_python_env/bin/python

/home/users/l/louedkhe/LSS_Surprise/DecodeYggdrasilFREMC.py

echo 'DONE'

Now, the next problem is the following from the log:

Running FREMC script on single CPU
/var/spool/slurmd/job7694693/slurm_script: line 20: /home/users/l/louedkhe/LSS_Surprise/DecodeYggdrasilFREMC.py: Permission denied
DONE

Why is permission denied?
Thank you.
Best,
Leyla

6

Hi, you are missing python in front of the Python script. Or you need to chmod u+x the script.

7

Thank you very much for all your help, it seems to work now.
Cheers,
L.