Network Outages and Maintenance

Cybera Network Status Page:

https://status.cybera.ca/


Outage and Maintance History:

Cybera Edmonton Maintenance Notification (5/15/2019)
Subject:          Cybera Edmonton maintenance.
Category:        Maintenance

Planned Start Time:   Friday, May 17, 2019, 21:00 MDT
Planned End Time:    Friday, May 17, 2019, 24:00 MDT

Cybera will be performing maintenance on its Edmonton router. Members should expect a few short duration outages (15 minutes) during the maintenance window. 

Contact: Please contact the Cybera Network Operations Center at NOC@cybera.ca or (403) 536-0880 when network problems are experienced outside of a maintenance window.


Emergency network maintenance – tonight (5/1/2019)

Category: Maintenance

Planned Start Time: Wednesday, May 1, 2019, 19:00 MDT
Planned End Time: Wednesday, May 1, 2019, 21:00 MDT

Anticipated outage duration: This maintenance work will result in three to four, 15 minutes or less outages during the time frame indicated above.

As a follow up to today’s outage, we require an emergency maintenance window later tonight that will result in a few short duration outages as we resolve a related outstanding configuration issue.

Early findings: The outage today was caused by a Routing Engine failure on our Edmonton router and a related software issue that compounded the outage duration. We are working on a full post mortem for this incident and will share our findings at a later time.

Contact: Please contact the Cybera Network Operations Center at NOC@cybera.ca or (403) 536-0880 when network problems are experienced outside of a maintenance window.


Network Outage (5/1/2019)

Subject:          Unscheduled network outage.

Category:        Network outage update

 

Please note that Cybera experienced a network outage today starting at 11:42. We are still investigating the scope and root cause of this outage. We are now investigating a hardware issue with our Edmonton Router and are working with the equipment vendor.

 

All IBG members as well as the VFS members who had redundancy configured have been redirected to Calgary. We are still investigating and will send periodic updates throughout the day.

 

Contact: Please contact the Cybera Network Operations Center at NOC@cybera.ca or (403) 536 -0880 when network problems are experienced outside of a maintenance window.


March 8 Outage Assessment (3/14/2019)

On Friday March 8, Cybera experienced an outage that impacted several of our members.

On Friday at 4:50 pm we began maintenance work on the DWDM in Edmonton to reset a backup power supply. This work was successfully completed by 5:00 pm. Shortly after (at 5:10 pm), we started receiving alerts notifying us that the DWDM was down. The main and backup power supplies failed, impacting members in Edmonton without regional redundancy. Peering, IBG (Internet Buying Group), VFS (Virtual Firewall Service), and shared hardware firewall services became unavailable to them. This outage continued until we were able to find and install a working power supply at 7:45 pm.

Once the DWDM came back online, the Peering and IBG services were restored for all members. Unfortunately, the DWDM outage triggered another failure. This impacted the affected firewalls, and spread to include the previously working VFS firewalls across both regions. After several hours of debugging, we determined there was a Layer 2 spanning tree loop issue. By 2:00 am on Saturday, the issue was resolved for many of our members.

We did not realize that, at that time, the Calgary region was experiencing similar issues. Later Saturday morning, some of our VFS members in Calgary let us know they were having issues (thank-you!). A review of the symptoms showed the same problems that Edmonton had experienced, and we were able to similarly resolve the issue by 10:00 am Saturday morning.

Two schools, one in Edmonton and one in Calgary, continued to have firewall issues. Our team worked with them throughout the day to debug, and were able to resolve their issues by late afternoon.

We are compiling a list of lessons learned which we will be reviewing next week. Once we’ve determined our next steps and how we will improve our procedures going forward, we will share that information with our members.

Edmonton DWDM issue and subsequent outage – resolved (3/8/2019)

Category:          Network failure

Start Time:        2019-03-08  05:06pm Local time.

End Time:          2019-03-08  07:45pm Local time.

Status update:

Service via Edmonton DWDM is restored. It was found out that the DWDM power supply failed and replacement is done.


Edmonton DWDM issue and subsequent outage (3/8/2019)

Category:          Network failure

Start Time:        2019-03-08  05:06 Local time.

End Time:          No ETA

Status update:

We are experiencing issue with our DWDM equipment in Edmonton and some customer connections which don't have redundancy via Calgary are affected. We will send status update as we get more information.

Mulugeta Yirba, Network Ops
Suite 200, 3512 – 33rd Street NW
Calgary, AB T2L 2A6
(403)210-5236, After Hours and NOC : (403)536-0880
A not-for-profit agency driving Alberta’s economic growth through digital technology.
SuperNet Outage (11/1/2018)
Category: Network service disruption on the Calgary SuperNet
 
Start Time: Thursday, Nov 1, 2018, 08:23 MDT
End Time: Thursday, Nov 1, 2018, 09:40 MDT
 
Duration: 77 minutes
 
This was a wide spread outage and service interruption to multiple services in the Calgary and aggregating communities. Axia dispatched a tech to the site, and the connection was then restored. Cybera is currently monitoring for service stability.
 
Contact: Please contact the Cybera Network Operations Center at network@cybera.ca or (403) 536-0880 if you have any questions.
 
 
 
Cybera maintenance (12/14/2017)
Subject: Cybera maintenance
 
Category: Scheduled Hardware upgrade for Google Cache Server (No down-time expected)
 
Start Time: Saturday, Dec 16, 2017, 9:00 MDT
End Time: Sunday, Dec 17, 2017, 16:00 MDT
 
Duration: 31 hours
 
During the time frame indicated above, Cybera will be performing hardware upgrade for Google Cache Server in Cybera. 
Google Services should not be affected as it is going to  re-route to another nearby Google Cache Server  but due to 
the nature of the work, a threat notice is being sent.
 
Note: We are planning a longer outage because Google is going to update content on the Cache server during the time which may
take a long period time.
 
Contact: Please contact the Cybera Network Operations Center at network@cybera.ca or (403) 536-0880 when network problems are experienced outside of a maintenance window.
 
 
 
Cybera Network Upgrade (7/14/2017)
Category: Scheduled maintenance (No down time expected)
 
Start Time: Thursday, July 20, 2017, 21:00 MDT
 
End Time: Thursday, July 20, 2017, 23:59 MDT
 
Duration: 179 Minutes
 
During the time frame indicated above, Cybera will be performing the software update on its Calgary core router. Cybera services should not be affected, but due to the nature of the work, a threat notice is being sent.
 
Contact: Please contact the Cybera Network Operations Center at NOC@cybera.ca or (403) 536-0880 when network problems are experienced outside of a maintenance window.
 

 
Cybera maintenance (6/15/2017)
Category: Scheduled software upgrade (No down-time expected)
 
Start Time: Thursday, June 15, 2017, 21:00 MDT
 
End Time: Thursday, June 15, 2017, 00:01 MDT
 
Duration: 180 Minutes
 
During the time frame indicated above, Cybera will be performing software upgrade on its Edmonton core router. Cybera services should not be affected, but due to the nature of the work, a threat notice is being sent.
 
Contact: Please contact the Cybera Network Operations Center at NOC@cybera.ca or (403) 536-0880 when network problems are experienced outside of a maintenance window.
 

AXIA Maintenance and/or Network Upgrade – Calgary Meet Me (4/26/2017)
Category: Scheduled Axia maintenance
 
Start Time: Wednesday, April 26, 2017, 00:01 MDT
 
End Time: Wednesday, April 26, 2017, 06:00 MDT
 
Duration: 60 Minutes
 
During the time frame indicated above, Axia will be performing maintenance on their network in Calgary. The outage duration should be less than the total duration of the reserved maintenance window and could include several small durations outages while maintenance is being done.
Cybera will not be performing any maintenance during that time and will be monitoring its connection to Axia.
 
Contact: Please contact the Cybera Network Operations Center at NOC@cybera.ca or (403) 536-0880 when network problems are experienced outside of a maintenance window.

Testing Redundancy for Google Service (3/17/2017)

Category:    Google Services

Start Time:    2017-03-17      8:00 am Local time.  (Friday)

End Time:    2017-03-17      4:00 pm Local times.

Anticipated outage duration: No Outage

During the time frame indicated above, CYBERA will be adding a redundant connection to Google Services at YYCIX ( Calgary) Internet Exchange. During the above outage window we do not expect any outage for Google Services, but if you experience any problem related to Google, please contact us via phone or email provided here.

Please contact the Cybera Network Operations Centre at network@cybera.ca or (403) 536-0880.


Shaw scheduled maintenance (3/11/2017)

Category:       Calgary Supernet Maintenance

Start Time:          2017-03-11    18:00  Local time.  (Saturday)

End Time:            2017-03-11    18:30

Anticipated outage duration: 10 minutes

During the time frame indicated above, SHAW will be performing maintenance on the Cybera to Supernet connection in Calgary. The outage duration should be less than the total duration of the reserved maintenance window and could include several small durations outages while maintenance is being done. Members connected via Edmonton PoP will not be affected.

Contact: Please contact the Cybera Network Operations Center at NOC@cybera.ca or (403) 536-0880 when network problems are experienced outside of a maintenance window. All outages, scheduled or not are posted here:


Cybera maintenance – tonight (3/8/2017)

Category:        EDMONTON router maintenance

Start Time:          2017-03-08  21:30  Local time.  (Wednesday evening)

End Time:           2017-03-08  21:45

Anticipated Outage Duration: 15 minutes or less

During the time frame indicated above, Cybera will be performing maintenance on one of its virtual core router in Edmonton. The outage duration should be less than the total duration of the reserved maintenance window and could include several small durations outages while maintenance is being done.

Contact: Please contact the Cybera Network Operations Center or (403) 536-0880 when network problems are experienced outside of a maintenance window. All outages, scheduled or not are posted here.


Axia/Supernet scheduled maintenance (Calgary) (2/8/2017)

Category: Calgary Supernet Maintenance 

AXIA Change Ticket # CA0001322541 

Start Time: 2017-02-08  00:01  Local time.  (Wednesday) 

End Time: 2017-02-08  06:00 

Anticipated Outage Duration: 5 minutes 

During the time frame indicated above, Axia/Supernet will be performing maintenance at the Calgary PoP. The outage duration should be less than the total duration of the reserved maintenance window and could include several small durations outages while maintenance is being done. 

Contact: Please contact the Cybera Network Operations Center at NOC@cybera.ca or (403) 536-0880 when network problems are experienced outside of a maintenance window. 


Axia/Supernet scheduled maintenance (Edmonton) (2/5/2017)

Category: EDMONTON Supernet Maintenance 

AXIA Change Ticket # CA0001322911 

Start Time:  2017-02-15  00:01  Local time.  (Wednesday) 

End Time: 2017-02-15  06:00 

Anticipated Outage Duration: 5 minutes 

During the time frame indicated above, Axia/Supernet will be performing maintenance at the Edmonton PoP. The outage duration should be less than the total duration of the reserved maintenance window and could include several small durations outages while maintenance is being done. 

Contact: Please contact the Cybera Network Operations Center at NOC@cybera.ca or (403) 536-0880 when network problems are experienced outside of a maintenance window. 


Network test message (6/9/2016)

Thanks for joining Revere, our home brewed notification system.

This page informs Cybera members of any network issues, problems, or simple notices such as maintenances. When real network issues occur, this page will be updated regularly with pertinent information.

Please add anyone within your organisation that should receive Revere network notifications messages. 

Expert Tip: You can bookmark this page and just link to it next time you receive a revere message.