Dear HPC Team,
seems like if set “o+rx” on /srv/beegfs/scratch/users/s/savchenk, after few minutes it changes back on its.
I also tried to use ACL, following my last question here, and it seems that similarly, the mask is set to prevent ACL.
What could it be?
I need to give access to shared data some another users. Maybe that’s not the right way to do it?
Best Regards
Volodymyr
Hi there,
For security reason, on Red Hat-based systems, ${HOME} directories are restricted by default to the user only (default permissions are 700). And we enforce this, on users’ ${SCRATCH} as well, given that the primary GID of all Baobab users is the same (for old users 1000:unige, for new ones 5000:hpc_users), which means that without defaulting to 700 anyone can access other users files.
To share data between users send us an email and we can then create a specific local group and then give to it access to a specific folder, /home/share/${PROJECT} or /srv/scratch/beegfs/shares/${PROJECT} .
NB , please also note that for easy sharing you need to set umask 0002 (thus new files and directories will be created with 660 and 770 permissions, respectively), otherwise you will be asked for confirmation every time you want to modify a file (see the example below).
Thx, bye,
Luca
Hi
thanks for clarification! It was indeed a bit confusing that the permissions seemed to work for a bit and then not. Is it described somewhere in the doc? Though for us, it is now perfectly clear!
we kind of work with public data/code, so really 755 (what I wanted to set) would be fine and equivalent to 750 (of course the really private files like authentication are actually private).
But solution with groups make sense of course.
I have mild concern that since we want exchange a substantial fraction of data, most of the work will end up in the shared space. Is it ok?
For the group, we previously discussed with @Pierre.Dubath that we would like to create a project group, if I understood in ISIs system (or at least with the same uids). There is an action pending about this.
Is this different from the local group you mention now? If so, do we need both?
Will the local group will used also on Yggdrasil?
Cheers
Volodymyr
Hi there,
You are right that there is no documentation (yet) about that, albeit on any IT system (even your personal machine) as a best practice you should not play with your ${HOME} permissions (some common software like ssh can complain with non-standard ones).
To be clearer: Baobab is a multi-user system, thus in general no data should be assumed “public” by default. Which is the reason why on shared group folders we avoid read/write permissions for others (the last cipher).
Yes, it is perfectly fine and other groups are already doing the same, either on /home/share (limited space) or /srv/beegfs/scratch/shares (plenty of space, but no backup at all).
In any case, data usage is not calculated per folder, but per user/group (cf. [howto] check disk usage on Baobab ).
On Baobab and Yggdrasil we use ISIs (thus AD) UID/GIDs whenever is possible, so if you ask for a shared space please tell us if you already have an ISIs/AD GID and/or if you need this group outside HPC (and we will ask for an ISIs/AD group for you).
Thx, bye,
Luca
Hi
You are right that there is no documentation (yet) about that, albeit on any IT system (even your personal machine) as a best practice you should not play with your ${HOME} permissions (some common software like `ssh` can complain with non-standard ones).
Good point!
It’s just that in my experience (on clusters of various sizes, and personal machines) its the permissions of Baobab which are the non-standard ones. Not excluding that some other systems might have the same approach as Baobab by default.
While 755 on home seemed quite customary, reasonable, and at least it is usually perfectly fine with ssh as you well know.
The thing which is unfortunate is that the Baobab home permissions are enforced in a just a little bit non-transparent and not documented way, which frankly costed us some time (though with little focus).
But if that’s the approach, very well, it is compatible with our needs, as long as we are aware of it.
On Baobab and Yggdrasil we use ISIs (thus AD) UID/GIDs whenever is possible, so if you ask for a shared space please tell us if you already have an ISIs/AD GID and/or if you need this group outside HPC (and we will ask for an ISIs/AD group for you).
Very well, I think we prefer ISIs GID then. Should we proceed with @Pierre.Dubath creating it, or is it possible (and should we?) ask the HPC team by email to do the same thing (with Pierre in CC, of course)?
Thanks!
Volodymyr
Hi there,
Are we talking about the enforcing (which is Baobab-specific) or the 0700 by default (which is a Red Hat-based distribution choice, cf. Best practices for umask in Red Hat Enterprise Linux - Red Hat Customer Portal and Chapter 4. Managing Users and Groups Red Hat Enterprise Linux 7 | Red Hat Customer Portal , not Baobab)?
FYI 0755 is the default on Debian-based systems, thus Ubuntu as well.
I am sorry for the inconvenience, we are now in the final step of ${HOME}-permission enforcing for all users and this is probably another confusion source. Just FTR, out of ~1’000 Baobab users, I recall less than 5 of them contacting us for such enforcing.
Given that ISIs management is not specific to HPC, please go on with @Pierre.Dubath and simply tells us the group name when asking for the shared space.
Thx, bye,
Luca
Are we talking about the enforcing (which is Baobab-specific) or the 0700 by default (which is a Red Hat-based distribution choice
Indeed! It’s quite clear redhat prefers 0700 (umask 077) even if umask by-omission is still 022. But it important to be safe on a multi-user server! Just that in my user experience on multiple academic clusters the home directories were always readable. Well, on one very large cluster they were not readable by default, but modifiable, instructions provided.
I am sorry for the inconvenience, we are now in the final step of ${HOME}-permission enforcing for all users and this is probably another confusion source. Just FTR, out of ~1’000 Baobab users, I recall less than 5 of them contacting us for such enforcing.
Indeed, Baobab-specific enforcement is a bit different!
I did not quite understand this paragraph, do you mean something will change/is changing in this respect, perhaps the way it is enforced?
Sorry to stress on this edge case, apparently.
Given that ISIs management is not specific to HPC, please go on with @Pierre.Dubath and simply tells us the group name when asking for the shared space.
I will proceed with this, thanks for the explanations!
Cheers
Volodymyr
Hi there,
Nothing will change, i.e. the enforcing will stay.
What is changing is how we (thus the HPC administrators) manage and monitor the cluster.
Thx, bye,
Luca
Alright! Would you say the change is something we the users will not even notice?
I have recently noticed the following effects, in fact I think they might be two distinct, not simultaneous, and possibly unrelated things: permissions on home changing to 711, and then home going unavailable:
[savchenk@login2 ~]$ ls -ld .
drwx--x--x 37 savchenk hpc_users 88 23 sept. 19:05 .
[savchenk@login2 ~]$ ls $PWD
ls: cannot access /home/users/s/savchenk: Communication error on send
[savchenk@login2 ~]$
Is the relation to the mentioned evolutions?
I just saw another user also experienced the second effect.
Regards
Volodymyr
Dear @Luca.Capello and colleagues,
@Pierre.Dubath has created for us the group:
GL_S_Astro_INTEGRAL
hence we can proceed with the creation of the shared directory.
Do you want me to send an email, or this message is enough?
Best Regards
Volodymyr
Hi there,
I prefer “not interesting for all HPC users” questions to be treated by mail or https://support-si.unige.ch , especially considering that the AD group is not enough for the shared space: we are missing the name, its members, if it needs to be on ${HOME} or ${SCRATCH}, etc.
Thx, bye,
Luca
Hi
of course, we would not want to bother other users!
But maybe a couple of small clarifications, possibly of general interest:
is not possible to use AD group name (perhaps, adopted with some convention) and members? I thought that’s why we create it with AD - to avoid a different, possibly incompatible source of this information. And I assume the same will be applicable to Yggdrasil.
that’s why you asked me to simply tell you the group name, right?
is the difference between home and scratch primarily in the backup policy, as far as we are concerned?
Regards
Volodymyr
Hi there,
ATM, Baobab (and Yggdrasil as well) only uses UID/GID from ISIs (as I wrote previously, cf. Some directory permission seem to change on their own - #5 by Volodymyr.Savchenko ), which means that we do not gets group memberships from ISIs.
Moreover, some ISIs group members could not have a corresponding HPC account, which is another reason I asked you about who on Baobab and Yggdrasil must be a member of such group.
Nope, there is also a space difference, given that on Baobab ${HOME} is on a 130TB BeeGFS share, while ${SCRATCH} on a different 1.2PB BeeGFS one.
Thx, bye,
Luca
Hello,
Very well, I understand now that Baobab and Yggdrasil get the numerical values of UID and GID but not the relation between them, the membership.
To me, this was not at all clear from the message you referenced! I had to infer one way or the other, and it seemed natural that the value of having the group defined outside of HPC appeared to be largely in it’s properties - the membership. The membership which could be controlled by the group representative, @Pierre.Dubath in this case.
But it seems that this is not possible, with every change in AD we have to ask for an update on Baobab, right?
And the relation needs to be established separately on Baobab and again on Yggdrasil? Or once for both Baobab and Yggdrasil?
While not all of our group users might have a Baobab account, all of those who have should be in the group at Baobab, with the GID derived from AD. Seems like very unambiguous approach, doing otherwise would introduce confusion, for us anyway.
But, perhaps, you have no access to the group content, so you do not know who is in it, and it needs to be communicated?
Ok! So, as far as I am concerned, there difference is primarily in the backup policy, and then also in space. I see no reason why not just always have both for a group, but who knows what requirements people have! I will recite this information separately in the request, after it is clear what impact will the information have. If we have to maintain two (or three?) different group memberships, we need to be a bit cautious.
Regards
Volodymyr
Hi there,
Let me clear once and for all: ATM, on the HPC clusters centrally managed (thus Baobab and Yggdrasil) the users/groups are local and not dynamically taken from the UNIGE AD/LDAP. The only dynamic value is the user password, which is not stored locally on the clusters, but lively checked against the UNIGE AD/LDAP.
Yes, ATM the HPC users/groups must be manually updated by the HPC team.
Please note that it is not a question of possibilities, it is simply a configuration choice that we have already started to reconsider because of new needs.
The user/group management is common to both clusters.
We have access to all the UNIGE AD/LDAP data, but we do not automatically synchronize the changes to the HPC infrastructure.
Thx, bye,
Luca
Hi
Thanks a lot for these explanations and a lot more details by email.
So, it seems possible to derive all information from AD group membership, with some manual synchronization actions, but how practically feasible it is is not entirely clear, even if it might be with the new needs.
And as you pointed out in the new doc, users are not advise to change directory permissions for personal scratch and home. Shared scratch belongs to root and is safe from this already.
Currently, users who missed documentation part about this, will notice their permissions reversed.
Cheers
Volodymyr