IV. Broadband Measurement

Using the open source tools and infrastructure provided by Measurement Lab (M-Lab), 16,696 Network Diagnostic Tool (NDT) measurements were collected from ACPS school connections over a two month period during the spring of 2016. Initial analysis confirmed that the district was using bandwidth caps for quality of service (QoS). QoS bandwidth caps are typically used to limit the total upload or download speeds for each device on a network in order to distribute the available bandwidth equitably. Having both capped and non-capped measurements provided a means to compare connections using QoS and non-QoS caps. Differences between bandwidth and latency were then examined, and finally other causes of latency such as upstream peering arrangements were explored. Latency measurements examine the quality of a connection, for example round trip time and packet re-transmission rate. Round trip time is the time it takes for a data packet to get from its origin to its destination and back, and packet retransmission rate is a percentage of how often data packets need to be re-sent to complete a web request.

Quantitative data analysis confirmed the presence of network Quality of Service (QoS) bandwidth caps on devices connecting to the wired network in classrooms. In the two charts below, the first showing download measurements faceted by classroom and school location, and the second showing upload measurements faceted by school location, there are caps in the measurements for all elementary school classrooms. Download speeds appear limited to ~100 mbps and upload speeds were limited to ~70 mbps.

That QoS limits were found to be in place on most classroom connections confirms that ACPS’ network management practices were successfully being applied. For this study, it means that most of the data collected were measurements of the network controls, not measurements of the total capacity of the connection. However, since data collected in two locations did not have bandwidth controls applied to all measurements, those data reflect the uncapped speeds of the district’s internet connection.

Values are in Megabits per second (Mbps), milliseconds (ms), or percentage (%) as indicated.

Source: “Broadband in Schools Github Repository - QoS Limited Numeric Summary,” Open Technology Institute.

Values are in Megabits per second (Mbps), milliseconds (ms), or percentage (%) as indicated.

Source:  “Broadband in Schools Github Repository - Non-QoS Numeric Summary,” Open Technology Institute.

These data show that for tests from non-QoS limited connections, the median speeds that ACPS’s wide area network gets are 234 mbps down and 283 mbps up. These findings emphasize that when designing a measurement initiative for assessing managed networks, an inventory of network constraints such as QoS limits should be conducted in advance, and the locations from which measurement tests are run should be chosen based on the intended measurement goals. As an example, a school might conduct measurements from a device connected to a switch near their internet connection, on an un-managed port, in order to collect baseline measurements of their network’s total capacity, and multiple measurement devices inside classrooms to measure different parts of the network with potentially different QoS controls.

The measurements comparing test data from QoS limited and non-QoS limited connections suggest that classroom connections (QoS limited), in addition to having throttled speeds, also have higher general latency. MinRTT is the minimum latency measured in transmissions from the server to the client, reported in milliseconds. In comparison, AvgRTT is the average latency of data transfers from the server to the client and is calculated as the sum of round trip times sampled during the test against the number of samples, also reported in milliseconds. Consequently AvgRTT is a latency measurement when the connection is under load during the test, and MinRTT is a latency measurement when the connection is not under load. In the numeric summaries in the previous section, we highlighted the median values for the latency measurements, Average Round Trip Time (AvgRTT) and Minimum Round Trip Time (MinRTT).

An interesting observation in the charts above is the clear clustering of tests when download or upload throughput is paired with MinRTT. When AvgRTT is compared to MinRTT (Average RTT/Minimum RTT for QoS Limited Download Measurements), there are similar clusters on both metrics, between 2-5ms and between 7.5-10ms. If a connection has decent speeds but high latency, the user experience of accessing content that requires low latency would be affected. For example, if a user was streaming a video, high latency on the connection would likely cause buffering, even if the speed of the connection was fast.

There are many factors influencing latency on the path to content, making cause difficult to assess. A school district purchases an internet connection, but how well the ISP is connected or “peered” with other ISPs can affect the user experience by introducing additional latency between the user and the content they are trying to reach.

The internet is an interconnected network of networks—all running common protocols that make the experience of browsing the internet relatively seamless. Home users, businesses, and schools purchase an internet connection from an ISP. These ISPs provide subscribers with access to the internet, by connecting their networks to transit ISPs. Transit ISPs provide connection to the internet itself, where most content we use as consumers is hosted.

To provide diversity in connectivity, M-Lab deploys multiple redundant “server pods” in most metropolitan regions, with each pod connected to a different upstream transit provider available in that region. In this way, examining M-Lab test data segmented by a named M-Lab server can show differences in measurements solely on the peering connectivity between the client ISP and the transit ISP connected to the M-Lab server.

By grouping our measurements by transit provider, the previous clustering is largely accounted for, as shown in the next series of charts. For QoS limited and non-QoS limited download and upload measurements, latency in the 7.5 - 10ms range is clustered in tests which ran through Level 3 connected M-Lab servers.

This finding suggests that at the time of testing, tests conducted between the ACPS network and Level 3 had higher latency than those tests conducted through other peered transit providers. It is also an indicator that the issues teachers or students experience when accessing online content, testing platforms, or other online services can also be caused by other factors outside of the school’s network and available bandwidth. The path to content and the hosting/peering arrangements of content providers and ISPs affect the delivery of content, notwithstanding the considerations schools make in provisioning their networks.