Problem with creating new environment

HPC Support HPC issues
1

Dear all,

I want to install PyTorch and torch_geometric in the new environment in Baobab. I used the following commands:

conda create --name new_environment python=3.8
conda activate new_environment
conda install pytorch torchvision torchaudio cudatoolkit=11.8 -c pytorch
pip install torch-geometric --upgrade

and the main.py file is:

import torch
from torch_geometric.data import Data

But when I wanted to run the sbatch script, I got the following error for the first line of the code (import torch):

ImportError: cannot import name 'elementwise_dtypes' from partially initialized module 'torch._prims_common' (most likely due to a circular import)

It seems that when importing torch, it will import the rest of the code which is ‘from torch_geometric.data import Data’. I attached the sbatch script.

#!/bin/sh
#SBATCH --job-name yourname # this is a parameter to help you sort your job when listing it
#SBATCH --error yourname.e%j     # optional. By default a file slurm-{jobid}.out will be created
#SBATCH --output yourname.o%j      # optional. By default the error and output files are merged
#SBATCH --ntasks 1                    # number of tasks in your job. One by default
#SBATCH --cpus-per-task 16             # number of cpus for each task. One by default
#SBATCH --mem 8GB                   #####CPU is mutually exclusive with the one for GPUs.#sbatch --mem-per-cpu=16000 # in MB
#SBATCH --partition shared-gpu         # the partition to use. By default debug-cpu. (shared-gpu)
#SBATCH --gres=gpu:1                    #sbatch --gpus=2
#SBATCH --time 0-11:59:59

module load Anaconda3
eval "$(conda shell.bash hook)"
conda activate new_environment
python3 -u main.py

I will be very thankful if you can help me with this issue.

Best regards

edit by admin: code highlight

2

Hello @Fatemeh.Nassajian

You are trying to install torch , is there any reason to not use one installed on cluster ?

(baobab)-[alberta@login2 ~]$ ml spider PyTorch/1.12.1-CUDA-11.7.0

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  PyTorch: PyTorch/1.12.1-CUDA-11.7.0
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Tensors and Dynamic neural networks in Python with strong GPU acceleration. PyTorch is a deep learning framework that puts Python first.


    You will need to load all module(s) on any one of the lines below before the "PyTorch/1.12.1-CUDA-11.7.0" module is available to load.

      GCC/11.3.0  OpenMPI/4.1.4
 
    Help:
      Description
      ===========
      Tensors and Dynamic neural networks in Python with strong GPU acceleration.
      PyTorch is a deep learning framework that puts Python first.
      
      
      More information
      ================
       - Homepage: https://pytorch.org/

Other version are also available:

(baobab)-[alberta@login2 ~]$ ml spider torch

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  PyTorch:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Tensors and Dynamic neural networks in Python with strong GPU acceleration. PyTorch is a deep learning framework that puts Python first.

     Versions:
        PyTorch/0.3.0-Python-3.6.4
        PyTorch/1.4.0-Python-3.7.4
        PyTorch/1.6.0-Python-3.7.4
        PyTorch/1.7.1
        PyTorch/1.11.0-CUDA-11.3.1
        PyTorch/1.12.0-CUDA-11.7.0
        PyTorch/1.12.1-CUDA-11.7.0
3

Hello,

Thank you for your reply. The reason to create the new environment is that I want to use “torch_geometric”, and there is no module “torch_geometric” installed on cluster.

Best regards

4

Dear @Fatemeh.Nassajian

Here the sofware installed: