02/22/2018 at 8:30 AM:
We are currently experiencing outages in the Olden, Ranger, Mineral Wells, Lingleville, Stephenville, Morgan Mill, Dublin, De Leon, Gorman, Bluff Dale, Brock, and Weatherford areas. Our teams are working to resolve this issue as quickly as possible.
Our support team will continue to reach out to you and all customers affected by this outage. Our network is passing traffic normally to an estimated 95% of subscribers. We have addressed some core routing changes over the past 24 hours and will make some additional changes in our maintenance window at 1AM – 3AM on 2/23/2018. This is being done out of peak hours to avoid disrupting most of our subscribers who will not be online. This change should restore services still experiencing outages or slow speeds this evening. The balance of issues that exist after this effort may involve issues with end-customer equipment damage that will have to be reset remotely or addressed with technician visits. Our support reps will be contacting those customers to remedy individual equipment settings or damaged equipment. Support is reviewing and responding to every customer that is messaging and calling us regarding this situation. If nothing can be done remotely, they will schedule a technician to come onsite. NextLink and our competitor’s residential agreements do not provide guarantees and credits for service disruptions, however, we will be providing a credit prorated based on your monthly rate and the downtime experienced. If you have a commercial link or dedicated licensed link we do have SLAs we adhere to move life critical businesses (first responders, ISDs etc.) to the top of our response list during these events. We do have dedicated teams working with you and all residential customers in parallel with our business customers. As a regular practice, we continue to identify and add resiliency daily. We have dedicated teams that do nothing but identify gaps or forecast potential issues and capacity planning efforts daily. This is not marketing speak or a an idle promise that we can’t deliver. The demand for our services and our 99% approval ratings over the past five years were not achieved by operating the way things have transpired over the past outages this month. Many will post on social media or give their opinion online about an isolated lightning strike, isolated fiber cut or power outage to a tower could not cause widespread issues with our network. Typically, this is true but over the past few outages when we restore the router and equipment at that location, it has caused route issues within our core. Our outage yesterday was due to multiple power surges at a Fiber regen location that eventually touches or feeds other wireless backhaul Links to serve other areas off our fiber ring. When the routers were rebooted, it caused a flood (backup re-route of traffic) of messaging to our core routers which spiked CPU utilization and caused “route flaps” (we have redundant routers in a quad configuration at the core and had to take one of them out that was causing disruptions to the network). Multiple circuits were coming back online at this location and caused this abundance of traffic into the core causing the ports to flap. When we experience these kinds of disruptions we know numerous customers will not come back online or come back to full speed on their own without first power cycling. The two prior outages this month were caused (not blamed) on other fiber circuits managed by us but owned by another provider that must make the repairs and once they were repaired, similar routing issues were experienced causing the much broader issue. Corrective action is underway to replace the existing core routers. Thank you again for your business, patience and understanding and we will soon “re-deliver” services worthy of the stellar reputation we reflected up to this month’s outages. Sincerely, Steve Burks, COO, Nextlink.
If you are still experiencing slow speeds or no connection after severe weather passes please power cycle your router by unplugging power for about 30 seconds. If after this you still have an issue, email email@example.com and we will get a ticket created to look at your radio and router and determine if we can resolve remotely or setup a time for a service call for you ASAP. Call volumes are high due to power outages and customer’s experiencing equipment damages so email is the best course of action to track and expedite any ongoing issue you have.
Weather (rain, ice, snow) will not affect a signal from our tower to your dish at home or business but severe wind, power surges to your equipment could cause damage or setting disruptions.
We will experience higher than normal call volumes and hold times during and after an outage. Please send an email to firstname.lastname@example.org to report issues. You may also leave a voicemail and or a “Call back option” when calling into support at 855-698-5465. Both the voice mail and call back option will hold your place in our call center queue to be sure your issue is addressed. We return these calls up to 10 PM CST and all others will be returned after 7 AM the next day so that we don’t disrupt your evening at home. Thank you for your patience.