CSE-IT supported websites that depend on a CSE-IT managed database, and CS UNIX accounts

During the CSE-IT maintenance window Sunday morning, 5/20/2018, a firmaware update resulted in some CSE-IT Virtual Server infrastructure becoming unavailable. Services affected include CSE-IT supported websites which depend on a CSE-IT managed database, and Computer Science UNIX accounts.

Resolved 2:27 p.m. 5/20/2018

[Major Incident] vole.cse.umn.edu – SSH connection shut down abnormally

At approximately 10AM today, CSE-IT began receiving reports of issues regarding  linux systems rebooting.  There was a problem with the scheduler for the task that was supposed to have rebooted systems at 4 a.m. this morning.  That reboot was announced last week.  A full postmortem will be done to find the exact cause of the problem.
We are very sorry for the problems this caused.


Planned Keller Data Center Outage

Most OIT and CSE-IT systems were down for maintenance on Saturday and Sunday.  CSE-IT Services that were affected included:

  • CS e-mail
  • CSE-IT Home Directories
  • CSE-IT Project Space
  • CSE-IT Databases and Websites
  • Networking

Maintenance has been completed.

Services that are stilll unavailable:

  • CS Samba Drive

Please report any additional issues that persist to csehelp@umn.edu

mysql database server

Our mysql server has stopped working a couple of times today. We are actively working to determine the cause and come up with a solution. If you notice any issues please report them right away. At 10pm we will briefly shutdown the servers to apply a patch.

Emergency Planned Outage

Due to a instability of the General Data Storage Service At 8:50AM today 10/8/2016 OIT Storage engineers will be performing an emergency shutdown of the General Data Storage system. Between 8:50 and 9:20 AM users will be unable to access files and folders stored on certain drives. The maintenance is being performed in order to clear the instability issues. Please contact Technology Help (1-HELP) with any questions or concerns regarding this scheduled maintenance.

Power outage

There was a brief power outage this weekend which affected buildings on the Twin Cities campus. CSE systems appear to be unaffected, but if you notice anything unusual, please contact us for further troubleshooting.

Virtual Infrastructure Disruption

We were experiencing issues with the storage for our Virtual Machines. This was causing wide-spread outages with various services, including Windows profiles and Unix home directories. The services have been restored. Let us know if you see any additional issues.
More information will follow in a post mortem.