Yesterday morning, we experienced a network outage that was the result of a fiber issue between two buildings in Los Angeles on Level3's network. This issue was immediately escalated internally and a review of our equipment showed no issues. Service technicians were dispatched on site in L.A. and they commenced checking light levels and identified an issue with connectivity at Level3. Level3 technicians were called on site to address the issue and escalated the matter to senior network engineers to address the problem internally. We are currently waiting on a report from Level3 to further address any faults in redundancy that may have allowed for this issue to occur. We will also be conducting a review of current circuits and looking to make configuration changes to avoid this type of disturbance in the future. Those change will likely include bringing on additional IP transit to further diversify our network. Although the occasional problem can occur on any network, we understand that this was abnormally long and that no down time is acceptable or ever comes at an ideal time. We very much appreciate your patience and understanding as this issue was addressed. We do not expect any further disruptions and will be taking additional measures to ensure nothing of this magnitude occurs again.
Wednesday, January 25, 2017