Fixed
The batch system should be back only but the submission process might still be slow for a while due to the generated backlog.
Reported
The batch system is refusing new job submissions. Investigating.
Grid-enabled distributed mass storage filesystem.
Job submission infrastructure for local users as well as grid experiments.
Interfaces for the users to access the cluster’s compute and storage resources.
Internal cluster network and its Internet access.
No incidents reported.
The batch system should be back only but the submission process might still be slow for a while due to the generated backlog.
The batch system is refusing new job submissions. Investigating.
No incidents reported.
No incidents reported.
We are facing an 'Xrootd storm". This is due to a lot of remote transfers initiated probably by "ignore locality jobs". We had to restart the mass storage services. Some of your jobs might have crashed.
No incidents reported.
It seems the batch system did not accept jobs anymore. The service was rebooted and is now working properly again.
m9 needed a complete re-install to fix the problem. /cvmfs should be up-to-date and same as other machines now.
To fix the issue with /cvmfs on m9, we need to empty the cache and reboot the machine. This will be done at 11PM today, Thursday 27/06, so please save your work and exit m9.
/cvmfs seems to be outdated on m9, compared to other machines.