Fixed
Migration of /pnfs headnode was successful, only CMS xrootd federation does not work yet (we are working on it).
Performances have been observed to be much better, but heavy NFS /pnfs usage inside jobs still impacts standard ls on /pnfs.
Reported
Dear Users,
As many of you have noticed, the usage of /pnfs has been complex lately. We have already taken several steps to mitigate this, but it has not been sufficient.
In light of this, we have decided to accelerate the migration of our central pnfs server to a brand new machine. It is in all ways a better machine (more cores, more memory, NVME for database performances), so we hope it will ease the slowness of the system.
For this, we will need to take /pnfs completely offline while the migration is being performed. We will therefore stop all /pnfs services tomorrow from Wednesday 00:00 to in the afternoon. We will inform everyone once the service is back online.
We are very sorry for all the troubles this brings to your analyses, and rest assured we are doing as much as we can to make everything stable again.
Cheers, The T2B IT Team