For the last few days, /pnfs is having performance issues, while some storage nodes can't serve files.
We have not identified the cause yet.
Mainly cd/ls becomes slow, and only a restart of the service makes it work fast again.
Also, a couple of storage nodes have a hard time serving files (hence why some of your files have issues), this usually solves itself after a while.
Cheers,
The T2B IT Team
Past Incidents
Sunday 8th October 2023
No incidents reported
Saturday 7th October 2023
No incidents reported
Friday 6th October 2023
No incidents reported
Thursday 5th October 2023
Mass Storage (/pnfs)problems on /pnfs
Since yesterday afternoon, we have been bombarded with xrootd requests from the CMS federation, causing /pnfs to be unstable. We are looking into possible ways to mitigate the problem.
The problem was fixed by increasing the memory given to the correct service.
/pnfs should be fully operational again.