File not found with Singularity

HPC Technical
1

Hi,
I try to run DeepVariant with Singularity, but the container doesn’t seem to find my files…

singularity exec --bind /home/users/username/scratch/ \
/home/users/username//scratch/deepvariant.simg \
/opt/deepvariant/bin/run_deepvariant \
--model_type WES \
--reads /home/users/username/scratch/TP_WES/proband.bam \
--ref  /home/users/username/scratch/TP_WES/Reference/hg19.fa \
--output_vcf  /home/users/username/scratch/proband.vcf

what didn’t work:
File not found, while present in the folder.

what was the error message:

[E::hts_open_format] Failed to open file "/home/users/username/scratch/TP_WES/proband.bam" : No such file or directory
2

Dear @Pauline.Sararols

The issue is that the directory scratch in your home directory is a symlink and that it targets a place not mounted. And in your case, your home directory is already mounted at the same location. If I specify an alternate location it works

(baobab)-[sararols@login2 Mut2]$ singularity exec --bind /home/users/s/sararols/scratch:/tmp/bla /home/users/s/sararols/scratch/deepvariant.simg bash

Singularity> ls /tmp/bla/TP_WES/Toy_dataset/Mutations/Mut2/Mut2ab_proband_final.bam
/tmp/bla/TP_WES/Toy_dataset/Mutations/Mut2/Mut2ab_proband_final.bam

and the .bam file is found.

If you don’t specify a bind option, the issue is that the symlink target a place not available from Singularity by default. If you add your scratch in the bind, it works and the symlink inside Singularity correctly resolves to the scratch space.

(baobab)-[sararols@login2 Mut2]$ singularity exec --bind /srv/beegfs/scratch/users/s/sararols/ /home/users/s/sararols/scratch/deepvariant.simg bash

Singularity> ls -la /home/users/s/sararols/scratch/TP_WES/Toy_dataset/Mutations/Mut2/Mut2ab_proband_final.bam
-rw-r--r-- 1 sararols hpc_users 377023952 Jul 14 14:50 /home/users/s/sararols/scratch/TP_WES/Toy_dataset/Mutations/Mut2/Mut2ab_proband_final.bam