Key points:
- Average Telstra DNS resolution speed is 23ms, 5 times faster than Google and 4.2 times faster than OpenDNS
- With over 10,000 discrete tests, run every 30 minutes for 7.5 months, the test observations are representative and the results are comprehensive
- Telstra's DNS infrastructure is not only way faster, it appears more reliable than both Google and OpenDNS too
All network engineers understand the importance of DNS. The reachability, performance and reliability of networks are all key operational metrics that are seriously impacted by DNS problems. Seriously. A lot of thought and engineering grunt goes into building and maintaining DNS operational infrastructure.
ISPs in Australia and elsewhere in the world, love to mess with their DNS servers to cheaply and easily implement redirection & blocking of target domains due to Government (or parental) censorship, alleged copyright infringement, whatever. A simple bypass for this basic blocking technique is often switching DNS servers, or resolvers, to a non-ISP public DNS server, like Google's 8.8.8.8 or OpenDNS's 208.67.222.222.
Internet users in many countries use this simple bypass method. But are there any downsides?
DNS name resolution speed is important so I thought I'd do some tests to see if there are any speed differences between say, my home ISP Telstra, and the Google and OpenDNS public DNS servers. One would normally expect the closest DNS server, your ISPs, to be the fastest due to physics but there is more to "fast" than just propagation delay.
It didn't take a lot of research to come to the conclusion that namebench, written a few years back by a Google engineer would probably be a good choice. It appears well regarded and will run happily on a headless Raspberry Pi computer running Linux, which is easily configured to take test measurements every 30 minutes or so, hour after hour, day after day, week after week.
I use Raspberry Pi (RPi) computers for many and varied purposes and this was a perfect opportunity to use one. I started with a standard headless RPi as I usually do. I live in Melbourne, Australia, my ISP is Telstra and I subscribe to a HFC Internet service with extra speed boost. I also use my Telstra cable modem/switch/router/firewall in switch mode or bridge mode or "NAT disable" mode. Goodness gracious - so many names for the same thing.
The net result of this configuration is that each of the 4 ports on my Telstra modem will give me a public IP address. Thanks Telstra! I have a Ubiquiti EdgeRouter Lite plugged into one port for my main home connection. The other 3 public IP ports are available for experiments, just like this one. By using a bridged connection like this with a public IP address assigned to the Ethernet port on the RPi, I give the test the best chance of accurate and uninterrupted measurements.
Experimental design
What exactly are the tests?
Every 30 minutes, namebench will try and resolve the IP addresses of the (Alexa) top 50 Australian domains with each of the 3 DNS servers (or resolvers) in the test. They are Telstra’s 61.9.133.193, Google’s 8.8.8.8 and OpenDNS’s 208.67.222.222.
That’s 150 DNS resolutions per test, every 30 minutes.
Preparing and running the test RPi
First, I created a headless Raspberry Pi.
Next, I SSH'd into the newly booted RPi and installed Python.
sudo apt-get install python-tk
Downloaded namebench.
wget https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/namebench/namebench-1.3.1-source.tgz
Extracted it.
tar xvfvz namebench-1.3.1
Changed directory to namebench.
cd namebench-1.3.1
Downloaded the (Alexa) top 50 Australian domains list.
wget https://raw.githubusercontent.com/synaptec/dns-test/master/top-50-domains-au.txt
Test ran namebench.
./namebench.py -q 50 -i data/top-50-domains-au.txt -O 61.9.133.193 8.8.8.8 208.67.222.222
That gave me an output something like this:
Then I configured a cron job to run it every 30 minutes.
Because I was using a public IP address, I added some extra security to the RPi to minimize the risk of it being subverted and penetrated. If you were to replicate this experiment behind your home Internet (NAT’ed) router, you wouldn't normally need to be concerned about adding that extra security.
Initially, back in early January this year, I was just going to run the tests for 2 or 3 days, enough time to get 100 test measurements. But I got distracted, forgot about it, and the experiment ran for around 7.5 months and executed just over 10,000 discrete test measurements.
Data analysis
You can check out the raw data and analysis at this GitHub repo.
Box plot
Cumulative frequency plot
Response time timeline
Based on the 95% confidence intervals, Telstra’s DNS mean response time is 5 times faster than Google’s DNS mean response time.
Based on the sample, the standard deviation for Telstra response times is 3-4 times lower than both OpenDNS and Google.
This means that Telstra's response times were very stable over the entire 7.5 months of the testing. OpenDNS and Google both had significant spikes/peaks which may indicate equipment maintenance, network conditions or other issues. The performance of both Google and OpenDNS improved in early June, which is interesting. Google's performance appears to degrade from early August.
