Contextualizing Crowdsourced Speed Test Data

Policymakers are interested in understanding the state of Internet quality at finer spatial granularity to recommend investments for Internet for All (and related initiatives). Crowdsourced speed test measurements, such as those collected by Ookla and M-Lab, offer a critical view of network access and performance from the user’s perspective. However, such measurements are sensitive to various factors, such as user’s subscription plan, local wireless network, user device type, etc. Using these measurements as-is without considering these factors can lead to misleading conclusions about Internet quality in a region. It is essential to contextualize these measurements to understand what the attained upload and download speeds genuinely measure.

In this work, we develop a novel Broadband Subscriptionn Tier (BST) methodology that identifies the subscription plan for existing crowdsourced speed test measurements. We use the Measurement Broadband America (MBA) to demonstrate that BST can achieve over 96% accuracy in inferring the subscription plan. Read the full paper here.

We augment approximately 1.5 million Ookla and M-Lab speed test measurements from four major U.S. cities with the BST methodology. We then show that many low-speed data points are attributable to lower-tier subscriptions and not necessarily poor access. Then, for a subset of the measurement sample (80 k data points), we quantify the impact of access link type (WiFi or wired), WiFi spectrum band and RSSI (if applicable), and device memory on speed test performance. Interestingly, we observe that measurement time of day only marginally affects the reported speeds. Finally, we show that the median throughput reported by Ookla speed tests can be up to two times greater than M-Lab measurements for the same subscription tier, city, and ISP due to M-Lab’s employment of different measurement methodologies. This work demonstrates the importance of contextualizing cRowdsourced speed test measurements. Based on our results, we put forward a set of recommendations for both speed test vendors and the FCC to contextualize speed test data points and correctly interpret measured performance.

You can find more details about this work in our recent paper, The Importance of Contextualization of Crowdsourced Active Speed Test Measurements.
This work received the Best Paper Award 🏆 at ACM SIGCOMM Internet Measurements Conference (IMC), 2022.