The complaint at the lake house in Muskoka was the universal one. “Why does the internet suck?”
The internet did not suck. The fiber line there is 2.3 gigabits in each direction. I confirmed this with a test. The gateway’s own wired speed test showed 2.31 down and 2.37 up. The internet was innocent. But each WiFi device in the house stopped at approximately 90 megabits. I tested different phones, different laptops, different speed-test servers, and both WiFi bands. The results were 87, 89, 92, and 90. A wired test at the same time gave 2,100.
Each test stops at 90
The identical results are the clue. Random interference gives random numbers. When each device stops at the same limit, one part of the path is that limit. True throughput of approximately 90 megabits is the exact signature of a 100-megabit Ethernet link. A 100-megabit link hid somewhere it did not belong.
One bad wall jack
I found the cause in the access point list. The in-wall access point connected its uplink at 100 Mbps, not at one gigabit. Gigabit Ethernet needs all four wire pairs in the cable. A link that connects at 100 usually has only two good pairs. Two good pairs is the classic signature of a bad keystone jack at one end of the run. The installer seated two pairs in that wall jack and called the work complete. The annual meeting of all cable installers, 1994: the theme for this year is two pairs. Two pairs is enough for anybody.
Twelve devices, one straw
The problem also had a multiplier. That access point carried 12 of the approximately 15 wireless devices in the house. The other access point had a good gigabit uplink. It carried three devices. So most of the household shared one 100-megabit straw. A 2.3-gigabit pipe sat behind the straw with no work to do.
The repair is not software. The repair is a punch-down tool and five minutes of work. I will terminate both ends of that run again, with all eight conductors this time.
The smart-queue detour
I must also report a detour, because it was a proper rabbit hole. Before I found the jack, I treated the problem as congestion. I applied a smart-queue traffic shaper. In the tests, the shaper looked like a hero. Latency under load decreased from 2.8 seconds to 284 milliseconds. The shaper also limited the 2.4-gigabit upload to 75 megabits, without a word. That repair is the same as a weld on a squeaky door. I removed the shaper completely. I also lost one hour to a bad speed-test endpoint. The endpoint reported a phantom 87-megabit upload. It sent me on a hunt for a line problem that did not exist.
Two rules survived
Two rules survived the trip. Rule one: judge an internet connection only from the gateway’s own wired test. Never judge it from a phone on WiFi. A speed test from a phone is the bathroom scale on carpet of networking. The WiFi is almost always the thing you measure. Rule two: when each device agrees on one suspicious round number, stop the tests. Find the link that is that number.
The internet was fine. This time, the correct first suspect was the wall jack.