How network utilization is measured

Recently I was asked how Cybera'€™s network utilization differs from traditional Internet Service Providers (ISPs).

The person asking the question quoted this excerpt from a 2013 research paper published by the University of California, Berkeley:

"Modern ISPs run their networks at a relatively low utilization. This is not because ISPs are incapable of achieving higher utilization, but because their networks must be prepared for link failures which could, at any time, reduce their available capacity by a significant fraction. Thus, most ISP networks are engineered with substantial headroom…"

First, I should point out  that I have never worked for an ISP directly, and my knowledge of the commercial sector comes from people who are in the industry. Second, there is no exact right or wrong way to utilize a network. Cybera'€™s motivations are just different from an ISP'€™s.

Network utilization is essentially the percentage of your total available bandwidth that is being used at a particular point in time. For example, if you have a 100 Mbps connection but you are only using 10 Mbps, then the utilization is 10%. From an ISP'€™s perspective, it'€™s economically advantageous to make full use of the available bandwidth by allowing as many users as possible to access the same line, up to its maximum capacity. However, a theoretical single connection that sees a high rate of utilization is not very redundant (i.e. without backup connections, if the line goes down, the users get no internet at all). Internet Service Providers tend to share the same amount of traffic load over two or more connections to increase redundancy, thus seeking to balance economics with risk of outages.

It'€™s important to remember that ISPs are driven by profit, and therefore try to run their network in the sweet spot (or optimum point) of allowing the maximum number of users while ensuring there is enough network space should everyone get crowded on one line. Their utilization may therefore appear low, but they are motivated to aggregate where they can. The utilization also depends on the design of the network and technology rollout. Many ISPs, including Shaw, recently upgraded their core networks to handle 100 Gbps, so I doubt they are short of capacity on the backbone, but their networks connecting to the core could be much, much smaller.

As well, it depends on if you are considering only the upstream (in most cases, Cybera plans for a lot of redundancy, which can create low utilization of our upstream) or the backbone network, where some links could see higher utilization than others.

Cybera carries two types of network traffic: Research and Education (R&E), and commodity internet. The pattern of use for these two types of network traffic are very different. The Research and Education network is very bursty by nature – sometimes the utilization is low and other times high. This makes it fairly unpredictable.

Although the average utilization can be relatively low over a long period, leaving enough headroom is still very important. Cybera strives to build in as much redundancy as it can, but CyberaNet (and its national backbone, the CANARIE Network) is essentially best-effort, with no binding Service Level Agreements. The headroom is absolutely necessary for research to happen.

On the commodity internet side, the traffic flow is constant and fairly predictable. So we can attain more aggregation without needing to upgrade capacity. Cybera purchases 5 Gbps from our upstreams, of which we generally use about two, so there is lots of spare capacity that could be used for growth or when one of the internet upstream connections goes down.

jf4

On the opposite end, ISPs will often over-provision at the local level because a street access point could share a fixed amount of bandwidth. This would cause the utilization to run very high at that level, but not necessarily beyond that point.

Of course, if you ask other networks about their utilization rates, the answer would be different. For instance, CANARIE does not carry commodity internet at all. So it's hard to have a fair comparison, as both the motivation and the characteristic of the traffic can be very different. Overall, I hope this helps put things in perspective in regards to the differences between the utilization of traditional ISPs versus Cybera'€™s network.