Dear Support Team,
I am having an issue connecting to the Bamboo cluster via VS Code Remote-SSH.
SSH access itself works normally, but the VS Code Remote-SSH connection hangs after successfully starting the remote VS Code Server. The Remote-SSH log reaches:
listeningOn==127.0.0.1:XXXXX==
...
<commit> : end
and then remains stuck at:
SSH tunnel message: Connected to SSH Host - Please do not close this terminal
I have checked the following:
-
Regular SSH connection to login1.bamboo.hpc.unige.ch works normally.
-
SSH authentication works with my Ed25519 key.
-
The remote VS Code Server starts successfully and remains alive.
-
The server listens on 127.0.0.1 on the reported port.
-
Manual SSH local port forwarding works.
-
For example, the VS Code server was listening on 127.0.0.1:36823, and:
ssh -N -L 50822:127.0.0.1:36823 bamboo
nc -vz 127.0.0.1 50822
successfully connects.
-
I also tested this after completely moving ~/.vscode-server aside and allowing VS Code to perform a fresh installation, with exactly the same problem.
Current environment:
- macOS
- VS Code 1.138.0
- Remote-SSH extension 0.128.0
- Remote OS: Rocky Linux x86_64
- Bamboo:
login1.bamboo.hpc.unige.ch
- User:
garnicac
Could you please help determine why the Remote-SSH client cannot complete the connection/handshake after the remote server has started?
I can provide the full Remote-SSH log if needed.
Best regards,
Josep
After further investigation, I found that several processes belonging to my user on login1 are stuck in uninterruptible D (disk sleep) state. For example:
PID State wchan
1898805 D __access_remote_vm
2029475 D __access_remote_vm
2885950 D __access_remote_vm
2915219 D __access_remote_vm
These are mostly ps processes that appear to have been spawned during the VS Code/Remote-SSH diagnostics. They cannot be terminated even with kill -9.
I also tested:
timeout 5 ps -fu "$USER"
which hangs and eventually times out, only returning my normal systemd --user processes.
This seems to indicate that there may be an issue at the login-node/filesystem/kernel level rather than with VS Code itself. This is also consistent with some recent problems I have experienced on Bamboo with Git processes returning Bus error/signal 7.
The original VS Code issue remains unchanged: SSH works normally, the remote VS Code Server starts and listens successfully, and the SSH tunnel can be established manually, but the Remote-SSH connection never completes.
Could you please check whether there is an issue with my session/processes on login1, or with the underlying filesystem/node, and whether the stuck D-state processes can be cleared?
I have not tried to kill the systemd user-session processes or otherwise terminate unrelated jobs.
Thank you for your help!
Hello, just wanted to say i confirm the exact same issues: VS code remote ssh never completing connection + ps -ef command hanging.
The ps -ef hangs indefinitely reading /proc/1457906/cmdlin. Is there a way to do something about this process PID 1457906 (owned by simionat)?
Dear users,
It seems some computation was launched on the login node, so I killed the process. login1.bamboo is now back in a good state.
Avg[|| 0.1%] Tasks: 357, 781 thr, 2409 kthr; 0 running
Mem[|||||||||||||||||||| 20.6G/504G] Load average: 45.44 2083.97 5703.49
Swp[|||| 78.4M/4.00G] Uptime: 7 days, 04:48:25
Could you please confirm that everything is working properly on your end?
Best regards,
Thanks a lot Gael. It works now on my side
Dear Gael,
For me the remote-ssh from vs code to bamboo is still not working, and is hanging when trying to read a thread (1839212) in D state. Indeed, the vscode remote cli looks for an already-running vscode server by reading the command line of every process and thread on bamboo node. And it blocks when it reaches that thread. It is still related to the same pid i mentionned earlier:
simionat 1457906 1 0 Sep17 ? 00:00:03 [codex]
Can you please kill it?
Thank you for your help!
Login1 has been rebooted, it should work again
I confirm it works! Thank you very much!