Some systems are experiencing issues

About This Site

Welcome on the T2B Cluster status page.

Please find status information about critical T2B cluster components, incidents and planned maintenance.

Mail subscription is available to get a notification when a component status change.

Stickied Incidents

Monday 12th August 2024

Mass Storage (/pnfs) Some issues with mass storage /pnfs [Rucio & Crab]

Hello,

Several users have reported that:

1/ Rucio does not allow copies to our RSE with error: Details: RSE excluded; not available for writing.

2/ Crab also complains that tasks can't be started because you are not allowed to write on your home directory on our site: Checkwrite Result: Unable to check write permission in /store/user/rougny on site T2_BE_IIHE

We are investigating both issues.

On the other hand, standard grid commands on your files (eg gfal-copy) seem to work without any issues.

Cheers, Romain

  • Dear all,

    After consulting with central CMS IT services, it seems that they have resolved the problem from their end. We also received confirmation from users that the rucio and crab indeed work as expected again.

    Kind regards,

    Olivier For the T2B Admin team

    • Past Incidents

    Saturday 23rd March 2024

    No incidents reported

    Friday 22nd March 2024

    No incidents reported

    Thursday 21st March 2024

    No incidents reported

    Wednesday 20th March 2024

    Mass Storage (/pnfs) pnfs slowness

    Hello,

    Unfortunately /pnfs is under heavy pressure from CMS global redirector. The server load goes above 30k, which slows everything down. Everythin works, but slowly ...

    We've stopped the redirector service for a few hours since yesterday, and while off everything works fine. We're trying regularly to make it work again, so if you find periods of slowness it is unfortunately unavoidable.

    We'll inform you as soon as the the situation is resolved.

    Cheers, The IT Team

  • Hello,

    We have found a fix and applied it to make sure the xrootd storm will not happen again and impact negatively all pnfs transfers.

    Cheers, Romain

    • Tuesday 19th March 2024

    No incidents reported

    Monday 18th March 2024

    No incidents reported

    Sunday 17th March 2024

    No incidents reported