Update @7:49 a.m.: Both servers are online now. ======================== Update @6:59 a.m.: The entire NetDepot Dallas datacenter is down and it looks like there is some sort of network/router issue. We are still waiting for an update from them. ======================== The huron server suddenly went off-line at 5;15 a.m. EST today (May 29). The hudson […]
UPDATE: The target of the DOS attack was a hacked WordPress website. The hackers created numerous files advertising Chinese webites, then directed traffic to those files. They were able to gain access to the site via a weak administrator password. The hacked site has been cleaned and traffic to the server has returned to normal […]
UPDATE – March 17 @ 12:02 P.M. EST – On March 6 we received the following maintenance notification from GNAX/NetDepot, one of the data centers we use to host our webservers: Maintenance Notification: Facility: Atlanta Time: 22:00 EST on Sunday March 15th. Window duration: 2 hours Anticipated impact: up to 1 hour Description: Core Network […]
UPDATE January 21: On January 7 the NetDepot datacenter experienced a power outage on the B feed in their Atlanta data center. This caused a number of servers to abruptly lose power and go offline. Once power was restored all but one of our servers successfully rebooted. The server that failed was one of two […]
UPDATE: The server performed an automatic disk check when rebooted. The disk check is complete and the server should be online shortly. The miami server failed to reboot after a kernel update this morning. We are investigating the cause now.
UPDATE @ 2:05 a.m. EST: Hypervisor 1 is online. UPDATE @ 1:56 a.m. EST: The datacenter has confirmed that this is a network issue and is working on resolving it. Hypervisor 1 is not accessible. It’s being reported as down by the nagios monitor, but some of the sites on the hypervisor are still accessible, […]
One of the RAID drives on the cherokee server is degraded and needs to be replaced. The drive replacement began at 5:30 a.m. EST and is expected to take no longer than one hour.
There is a network issue at the datacenter that’s affecting Internet connectivity to the host1 and superior servers. The servers are up they are just not accessible via the Internet. We are waiting for a response from the technicians at the datacenter regarding the exact issue and ETA on resolving the issue.
UPDATE @ 2:47 p.m.: The /home partition became read-only again before we could begin moving sites. We are moving the backup drive from the old server to the new server and will begin restoring sites from the backup drive. The host2 server has crashed. It was rebooted and is performing an automatic disk check. Once […]
Earlier today we sent an email stating that the mackinac server needed a file system check on the /usr partition. After examining the drive, the technician at the datacenter suggested that we check the entire drive. I have given him the go-ahead for this. The file system check for the entire drive should take about […]