Small Utility Bash Functions for HPC Tasks

HPC Technical
1

Hello everyone,

I’ve created a set of small bash utility functions that might be useful for those working with HPC tasks:

  1. tailast: Tails the most recently modified file in a given directory.
  2. watchjob: Watches the status of a specific job by its job ID.
  3. git_sbatch: Ensures that an sbatch submission uses the script from the exact git commit from which it was queued, preventing any discrepancies due to git updates after queuing.

You can find the details and the code in this gist: https://gist.github.com/albertbuchard/3fadf04298ddb8a9c255e0c10c6d997c

I hope you find them helpful. If you have any suggestions or improvements, please feel free to share!

2

Hi @Albert.Buchard thanks for sharing, this is appreciated.

my two cents:

  • in your watchjob script: please use a value for -n bigger than 1 sec, for example 10 sec or even more.
  • in git_sbatch you may be able to (maybe) simplify the script by using the --wrap option of sbatch In this case you don’t need a temporary file but you submit the script as a string.

Final suggestion: if you need an UNIGE git instance, we are maintaining a central instance of Gitlab.

1 Like
3

Thank you for the feedback @Yann.Sagon ! I did not know about the --wrap param, I’ll check it out.

We’re using UNIGE’s Gitlab for our projects - but gists are good for small stuff like that :wink:

4

Or you can use snippets on GitLab :upside_down_face:: https://gitlab.unige.ch/-/snippets/171

Best

1 Like