username: moinatl
Dear HPC team,
I wanted to launch a simulation that I do as usual, and it was working perfectly before the maintenance. This morning I wanted to launch a new simulation, and I got this error:
A requested component was not found, or was unable to be opened. This
means that this component is either not installed or is unable to be
used on your system (e.g., sometimes this means that shared libraries
that the component requires are unable to be found/loaded). Note that
PMIX stopped checking at the first component that it did not find.
Host: cpu046.bamboo
Framework: gds
Component: shmem2
[cpu046.bamboo:3447299] PMIX ERROR: NOT-FOUND in file client/pmix_client.c at line 562
[cpu046.bamboo:3447299] OPAL ERROR: Not found in file ext3x_client.c at line 112
It looks like pmix_init failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during pmix_init; some of which are due to configuration or
environment problems. This failure appears to be an internal failure;
here’s some additional information (which may only be relevant to an
PMIX developer):
pmix_gds_base_open failed
→ Returned value -46 instead of PMIX_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
The application appears to have been direct launched using “srun”,
but OMPI was not built with SLURM’s PMI support and therefore cannot
execute. There are several options for building PMI support under
SLURM, depending upon the SLURM version you are using:
version 16.05 or later: you can use SLURM’s PMIx support. This
requires that you configure and build SLURM --with-pmix.
Versions earlier than 16.05: you must use either SLURM’s PMI-1 or
PMI-2 support. SLURM builds PMI-1 by default, or you can manually
install PMI-2. You must then build Open MPI using --with-pmi pointing
to the SLURM PMI library location.
Please configure as appropriate and try again.
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[cpu046.bamboo:3447293] Local abort before MPI_INIT completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
What does it mean exactly?
Thank you for your answer!
Laure