This pilot study in Alexandria City Public Schools identified several key findings for researchers to consider when planning future work on measuring broadband speeds in public schools. First, it is clear that school districts need affordable options for network measurement tools—there is a broad need for open tools that can meet the demands of public school networks. To solve this problem, however, developers must first understand the complexities of network measurement in schools, taking into account any network management practices that school administrators use. Beyond network measures, network administrators will also have to account for potential upstream “interconnection” arrangements that could be affecting network performance. Finally, in order to make informed decisions about the network, there must be a clear feedback loop between data on overall network capacity and performance measurement in the classroom.
V. Measurement Findings
Photo: Pexels
1. School districts lack network measurement tools
The prototype system to measure network capacity at the point of use in the classroom was validated by ACPS, confirming that tools to provide this data meet an unfilled need. ACPS staff also brainstormed other ways that they could use the pilot system, beyond the tests conducted in this pilot. For example, they suggested running 4-5+ tests concurrently in the same classroom on the same network to see how many concurrent tests could run in a given location before their network showed signs of high load. As future projects are developed, some valuable additional features to the prototype would allow future partners to integrate network capacity or utilization data from vendor systems. For example, network switches use a common protocol called Simple Network Management Protocol (SNMP) and an SNMP plugin for our system could be integrated to receive switch utilization data. Similarly, a vendor system plugin could provide data on WiFi utilization. These additional data could be displayed alongside performance measurements to provide a more nuanced understanding of performance, capacity, and utilization.
2. School networks present unique technical challenges for network measurement
This pilot study provided an opportunity to prototype new tools for running automated network performance tests from multiple vantage points within a network. The proof of concept measurement prototype used Internet2's NDT command line client, running on a small computer connected to the school network. An important goal of the pilot was to prototype a measurement system consisting of a central server to provide scheduling and data collection from one or more devices within the school network. Placing small, unobtrusive devices in different places within a school network allowed us to measure at several different points, including the point of student use, simulating a computer in the library, a wireless laptop or tablet, or a desktop computer in the classroom. Uncovering many of the challenges of measurement in actively managed school networks will help inform future iterations of a measurement system.
3. Network Management Practices Should Be Considered In Any Measurement Program
The initial analysis found that Quality of Service (QoS) network management rules had placed bandwidth limits on most classroom connections. This is normal networking practice in a large institution, but for the purposes of the study this meant that most of the data consisted of measurements of the network controls, not measurements of the total capacity of the connection. The result does confirm that ACPS’ network management practices were successfully being applied. However, understanding where these controls are in place when determining the points from which measurement takes place is paramount to measuring what one intends to measure. In this study, measurements from both QoS limited and non-QoS limited connections were collected, which was useful for comparison.
4. Upstream ISP peering may affect school network performance
The analysis found clusters of tests with higher and lower latency and higher and lower bandwidth measurements. To understand the potential cause of this clustering, the data was segmented by transit provider. By segmenting the data this way, the analysis showed that tests conducted between the ACPS network and Level 3 had higher latency than those tests conducted through other peered transit providers. Transit providers connect networks of ISPs to the Internet itself, through “interconnection” or “peering” arrangements. Because M-Lab hosts its servers in the data centers where these transit provider to ISP interconnections are made, the data collected at ACPS can be segmented by different transit providers.
This finding indicates that the issues teachers or students experience when accessing online content, testing platforms, or other online services can be caused by factors beyond the control of the school, like its network and available bandwidth. The path to content and the hosting/peering arrangements of content providers and ISPs also affect the delivery of content, notwithstanding the considerations schools make in provisioning their networks.
5. Performance measurements should be compared with data on network capacity
It’s clear from interactions with ACPS staff and other educators that simply measuring performance of a network connection, whether at the classroom and device level or simply from the egress point of a school network, only provides part of the picture of whether a network is functioning at needed capacity.
ACPS showed the New America team the vendor systems they use to provision and manage WiFi access points, as well as the utilization graphs of network hardware. Our partners at ACPS could see total bandwidth used over time from a particular WiFi access point, and had data on the total network utilization over time at the district and school levels. But connecting that data from disparate tools and with speed/performance measurement data from client tests is no small task. The pilot project identified the need for additional features to the prototype measurement system,including the ability to interface with commonly used network management protocols like SNMP, as well as support plugins for vendor provided systems or at the very least provide data import capabilities. Client measurements AND utilization capacity of the network would help to demonstrate how client measurements relate to overall capacity.
