Shortly before 10:00 AM, Wednesday, July 29, 2009, LAWN access became unavailable for the CCB. It was restored to service at approximately 11:05 AM, July 20. OIT is investigating the cause. Additional information will be posted as it becomes available.

Update:
-----Original Message-----
From: System Operator [mailto:operator@fantasy.oit.gatech.edu]
Sent: Wednesday, July 29, 2009 2:46 PM
To: oit-availability@lists.gatech.edu
Subject: Unplanned Network Service Interruption on 07/29/2009 - 0940 to 1130


Unplanned Network Service Interruption on 07/29/2009 -
Outage was from about 09:40am to about 11:30am


WHO WAS AFFECTED BY THIS INTERRUPTION?
All users of the general Georgia Tech network may have been affected. The
affect was networking sluggishness / lack of connectivity to several
campus locations and services, including the LAWN wireless network.

WHAT HAPPENED?
Network connections to & through central site services were hindered
because of hardware on the network interfering with normal network
traffic. Even though the hardware in question was found and isolated, the
network team is working to correct the problems caused by this
device. Our Network Backbone team is closely monitoring the network in
case the problem returns.

HOW LONG WAS THE SYSTEM OR NETWORK DOWN/SLOW?
The problem began at about 9:40am and cleared up by 11:30am.

WHAT DO YOU NEED TO DO NOW THAT THE SYSTEM IS AVAILABLE AGAIN?
Business as usual. No follow up action should be necessary.

WHAT IF YOU HAVE QUESTIONS?
If you still have questions, please contact your departmental IT support
staff, or the OIT Technology Support Center at 404-894-7173. For after
business hours networking trouble, please contact the Network Operations
Center (NOC) at 404-894-4669.

TECHNICAL DETAILS ABOUT THIS UNPLANNED SERVICE INTERRUPTION
The DNS appliance was handing out routes incorrectly and diverting some
network traffic. This problem was found in the LAWN network, locations in
Skiles and Architecture and off campus network traffic. The offending
network service was disabled around 10:57 today and testing and cleanup
continued until 11:30 . Traffic routing returned to normal, but our
campus networks are still being monitored very closely.

Owner of Alert
TSO