Hi, It appears that several nodes are currently in drain/down state. [image] Regards, Debajyoti

Dear @Debajyoti.Sengupta Here the status of this incident: [image] [2023] Current issues on HPC Cluster HPC ChangeLog Baobab infiniband issue Dear users, Due to an incident that occurred last night on the infiniband fabric on the Baobab cluster. Compute nodes…

They continue to be drain/down [image]

Dear @Debajyoti.Sengupta We have identified the root cause of the issue. An infiniband switch is experiencing hardware malfunction, resulting in interference with the infiniband traffic across the entire cluster. Currently, the nodes connected to this switch have been temporarily disabled and power…