Questions for using HPC

HPC General
1

Dear HPC community,

I would like to use the cluster for image processing tasks (at least benchmark algorithms). I developed numerous image processing algorithms compatible with the adaptive particle representation and I’d like to test them on big data-sets (a few To, or less if it is too much).

For the pipeline to run I need the C++ LibAPR library and its dependencies to be installed. Is this possible and how can I do it?
The full pipeline is controlled with Python and requires a few external packages (mainly for the binding with the C++ library and for machine learning). Is this also possible to run such a pipeline on the cluster?

Finally we are making the LibAPR library working on GPU using cuda. Most of the processings could therefore be done on the GPU in the near future. Is this also possible to run such a pipeline on GPU nodes?

I am not familiar with HPC computing, I used SLURM once and I never set the dev environment, hence my newby questions.

Thanks for your help and have a nice day!
Jules

2

Hi,

I just found this unanswered topic by browsing the forum, not sure it is still relevant.

We already have APR on the cluster:

[sagon@node025 ~] $ ml spider APR

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  APR:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Apache Portable Runtime (APR) libraries.

     Versions:
        APR/1.6.3
        APR/1.7.0
     Other possible modules matches:
        APR-util

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  To find other possible module matches execute:

      $ module -r spider '.*APR.*'

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "APR" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider APR/1.7.0
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------

If the pipeline control is a very lightweight process, you may launch it on the login node. If not, you should submit it as a job.

Best