CoC Chiller Failure

At approximately 6:55 PM on August 18, 2008, TSO became aware that power had failed to the chiller units in the CoC server room. As a result, all CoC servers housed in that server room are being shut down due to heat. Affected services include Zimbra email, CoC web sites, UNIX and Windows authentication, Filemaker Pro databases, home directories, managed file systems, and RT ticketing. CoC Exchange, SharePoint, and BES services should not be affected. Additional information will be provided as it becomes available. Any email received during this time will be queued and delivered once the servers have been restored to operational status.

Update: The chillers were brought back online at approximately 8:00 PM on August 18. Many of the servers had yet to be shut down when cooling was restored. After the server room temperature had returned to normal, TSO began turning the remainder of the servers back on. Another update will be posted when all services have been restored.

Update: All services have been returned to normal operation as of 9:50 PM, August 18.