The main issue now is their continued use of port 8080 and I still wonder why they chose in the first place over the default HTTP like what most websites use and TestMy has no problem using. Even Ookla’s new HTML5 beta uses port 8080, which leaves it open to traffic prioritisation abuse, i.e. Had they used port 80, this would present a significant challenge for the ISP to manipulate speed test traffic as it would no longer be as simple as giving traffic priority over a certain port number.
Even if the affected ISPs in the article stop prioritising port 8080, who knows what other ISPs are affected and how long the affected ISPs got away with delivering inflated test results, particularly when we identified 4 ISPs (two broadband and two 4G mobile operators) that gave significantly quicker throughput over port 8080 compared to port 80.
Finally, even without any traffic prioritisation or throttling between port numbers, there is one other thing to watch out for - If your ISP hosts a Speedtest server, then there is a good chance it will use that by default unless you manually choose a server to use, in which case the speed test traffic never meets the Internet. Whoops.
On the other hand, it does make a pretty effective tool to see roughly how fast your link is between your network and the ISP itself, particularly if you don’t have access to the link status (e.g. ISP DSL router bridged to the WAN port of another router.) For example, if speedtest.net gives an unusually low result, then there’s a good chance there’s a problem with your router or the link to the ISP, whereas a slow result with TestMy or your own test server indicates one or more congested/slow routers along the route as Xercus pointed out.