Why You Shouldn’t Trust Network Speed ​​Tests

The media is filled with hyperbolic claims that “Our network is the fastest!”

And there are many so-called “Speed ​​Test” tools available on the internet. Most run easily in a web browser.

Should these tools be trusted?

Not really.

Popular speed test tools provide a very narrow and limited measure of network “speed”.

It’s entirely possible that a network labeled as “fast” may actually perform poorly for many applications.

Why is this so?

What’s in these speed tests?

Most internet speed test tools run a limited regimen of tests:

  • ICMP Echo/Reply (“ping”) to measure round-trip time (although most tests are unclear whether they report round-trip time or halve it to estimate one-way latency).
  • HTTP GET (download) and PUT (upload) to measure TCP bandwidth.

Some more sophisticated tools can add things like:

  • Traceroute (correctly done with UDP packets, poorly done with ICMP Echo packets)
  • DNS queries

Some speed test tools use IPv4, some use IPv6, some use whatever the underlying web browser and IP stack chooses.

Sounds good, so what’s wrong with them?

Network performance is closely related to how devices in a network communicate with each other.

For example:

  • Does the application software (and its server) use UDP or TCP? UDP is vulnerable to many network phenomena such as IP fragmentation, highly variable latency/jitter, packet loss, or packet sequencing corruption (i.e. the sender sends the packets A, B and C, the recipient receives them in the order B , C, A.), etc. TCP, on the other hand, while reliable, can hold back data delivery to the receiver while it internally tries to deal with packet loss, end-to-end latency changes, and network congestion.
  • Does the application data have real-time constraints? For example, voice or video conferencing applications have very strict time constraints, otherwise images may break, freeze or words get lost.
  • What is the size of the data blocks sent? Larger data, especially very large high-definition video, is more vulnerable to network loss, transient congestion issues, or IP fragmentation issues than smaller data packets.

The bandwidth number generated by most speed test tools is based on World-Wide-Web HTTP GET (download) and HTTP POST (upload) transactions. These are bulk transfers of large amounts of data over TCP connections.

Bandwidth numbers based on TCP bulk transfers tend to be good indicators of how long it takes to download a large web page. But these numbers can be poor performance indicators for more interactive applications (eg Zoom).

Additionally, TCP tries to be a good citizen on the network by doing its best to avoid contributing to network congestion. TCP contains several algorithms that fire when a new connection is started and when congestion is perceived. These algorithms cause the sending TCP stack to reduce its transmission rate and slowly ramp up to full speed. This means that every new TCP connection begins with a “slow start”. Additionally, any lost packets or changes in perceived round-trip time can send the sending TCP stack into its congestion avoidance regime during which traffic flows will be reduced.

Modern web pages tend to be filled with a large number of subsidiary references. Each of these tends to spawn a Domain Name System (UDP) lookup and a new TCP connection (each with its own slow start penalty.) Therefore, the performance of modern web pages is often not so much limited by network bandwidth as by protocol algorithms and network round-trip times.

So what do we really need?

Unfortunately, a comprehensive measure of network path quality and speed includes a large number of often obtuse numbers.

  • If the path contains parallel elements due to load balancing or hard link binding. (In other words, it’s good to know if all traffic follows the same path or if it’s split across multiple paths with possibly very different characteristics.)
  • Whether the network path is symmetrical or whether each direction takes a different route. (It’s very common.)
  • Path MTU (Maximum transmission unit for the entire unidirectional path – a separate value is needed for each direction.)
  • End-to-end latency, and often, more importantly, a statistical measure of the packet-to-packet variation of this delay, often referred to as “jitter”.
  • Packet loss rates and a measure of whether this loss occurs continuously or in bursts. (This is especially important on paths that include technologies prone to outside interference and noise such as wireless links.)
  • Buffering along the path (in other words, if the path might suffer from “bufferbloat”.)
  • Packet resequencing rates and a measure of whether it is burst or continuous behavior.
  • Whether there are “hidden” proxy devices (most likely HTTP/HTTPS or SIP proxies) relaying the traffic.
  • Indicates whether there are rate limiters or data quotas on the path.

What can a user do?

Users are somewhat limited in their ability to control protocols and applications.

The user can check the following:

  • If you are using Wi-Fi at home or at work in conjunction with Bluetooth, make sure you are connected to Wi-Fi on the 2.4 GHz band. Many user devices have only one radio. If this device is connected to wi-fi in the 5 Ghz band, then this radio quickly switches between Bluetooth on the 2.4 Ghz band and wi-fi on the 5 Ghz band. This is a recipe for generating destructive packet loss and jitter.
  • Make sure your home Wi-Fi and router devices have the new anti-bufferbloat code. See What can I do about buffer bloat?
  • Know when you are sharing your network resources with other users or other applications.

What tools do developers have to ensure apps perform well under real-world conditions? Enter the network emulator.

Speed ​​test tools tend to give an optimistic report of network behavior for a very limited number of applications. Similarly, many network developers only test their code under optimal laboratory conditions.

There are tools available to developers so they can ensure that their code and products are robust and perform well in the face of unavoidable sub-optimal network conditions.

Most of these tools fall under the heading “network emulators”. These effectively act as a troublesome middleman, delaying some packets, dropping others, perhaps resequencing packets, or even duplicating them.

Network emulators offer a variety of capabilities and precisions:

  • Simple emulators are built into some mobile phones.
  • There are a few open source packages that usually exist as kernel modules for Linux or FreeBSD. These usually have to be used through an obscure command line interface. And their accuracy can vary wildly depending on the underlying hardware.
  • Some external devices are inserted into an Ethernet link (as one would insert an Ethernet switch). These devices tend to have better accuracy and performance and often have web-based graphical user interfaces. IWL’s KMAX is in this category.

There are also mathematical emulators. These are more for those who are designing large networks and want to perform a queuing theory analysis of how that network might perform if new links are added or removed.

This post was originally published on IWL.

Kevin M. Risinger