doc_loomis
Well-known member
Hi guys.
As many of you, including myself, experience issues with the slow loading website, I did some investigation. It seems it's related to the IPv6 endpoint.
Many of you will be using dual stack with both IPv4 and IPv6 enabled. ICMAG also offers both, IPv4 and IPv6 endpoints.
Now, the is a technique in dual stack that will send requests to both the IPv4 and IPv6 endpoint and pick whichever answers first, slightly preferring IPv6. I don't want to go to deep into detail, for the interested google for "happy eyeballs".
Now, when we hit the v6 endpoint, it is incredibly slow, as the curl output below shows (it might seem a little cryptic if you're not working in IT, but I guess the admins will understand). As some of you might wonder why this happens only sometimes, DNS has a caching mechanism. It prevents clients from asking the DNS server for an IP over and over, when those IPs usually don't change very often.
So the owner of a domain, in this case icmag, can set a so called TTL (time to live). This is set to 300 seconds for icmag.com. So if you're lucky with your first request to icmag.com, and you hit the IPv4 endpoint, you'll have 5 minutes of joy. If you hit the IPv6, you'll have at least 5 minutes of poor performance. Until the TTL on your local machine expires, the client asks for DNS resolution again, and so on.
See the 300 in the answer section, that's the TTL.
IPv6
Here's a request to the IPv6 endpoint, notice the --resolve option to curl, and the Connected to line. Also, the third to last line, which shows the total time it took (20 seconds). I edited out the uninteresting bits.
IPv4
The same request to the IPv4 endpoint takes only half a second to complete:
Workaround
I disabled IPv6 and the website is fast and snappy. But that's not a desirable solution, obviously, so only suited for a quick test. If you're a linux user, you can disable IPv6 for an interface with (as root, and obviously change the wlp0s20f3 to the identifier of your interface)
This change is not persistent, and will be gone after a reboot. You can google how to make it persistent, if you like.
The less invasive option is to add a hosts entry to your /etc/hosts, like:
That's also suitable for Mac users. That's what I do right now, with the downside, that you'll have to remember it in case the IPs do change for some reason.
I think adding a hosts entry is also possible in Windows, but I haven't touched it in 20 years, so I don't know how to do it there.
*edit* HERE is a howto for windows users.
Explanation
If you add a hosts entry, it will tell your computer to bypass the DNS lookup, and use the IP you configured to resolve requests to icmag.com, thus not seeing the IPv6 and not hitting the IPv6 endpoint. This is the least invasive option for a workaround. But note the downside: You will have to remember it, and you will have to edit it in case the IPs for icmag change. Usually this doesn't happen very often, but it might.
@admins of icmag
I'm a devops/cloud/sre freelancer and happy to assist in debugging, if necessary. Feel free to reach out. It would be nice to find a solution for this, as disabling IPv6 isn't desirable and I couldn't find an extension, that will disable either happy eyeballs or v6 DNS lookups for a certain website. So for now, a static /etc/hosts entry it is for me :/
EDIT
I have noticed this issue on different websites using cloudflare in the meanwhile as well.
Also, here's an issue report on the cloudflare forum.
As many of you, including myself, experience issues with the slow loading website, I did some investigation. It seems it's related to the IPv6 endpoint.
Many of you will be using dual stack with both IPv4 and IPv6 enabled. ICMAG also offers both, IPv4 and IPv6 endpoints.
Now, the is a technique in dual stack that will send requests to both the IPv4 and IPv6 endpoint and pick whichever answers first, slightly preferring IPv6. I don't want to go to deep into detail, for the interested google for "happy eyeballs".
Now, when we hit the v6 endpoint, it is incredibly slow, as the curl output below shows (it might seem a little cryptic if you're not working in IT, but I guess the admins will understand). As some of you might wonder why this happens only sometimes, DNS has a caching mechanism. It prevents clients from asking the DNS server for an IP over and over, when those IPs usually don't change very often.
So the owner of a domain, in this case icmag, can set a so called TTL (time to live). This is set to 300 seconds for icmag.com. So if you're lucky with your first request to icmag.com, and you hit the IPv4 endpoint, you'll have 5 minutes of joy. If you hit the IPv6, you'll have at least 5 minutes of poor performance. Until the TTL on your local machine expires, the client asks for DNS resolution again, and so on.
See the 300 in the answer section, that's the TTL.
Code:
$ dig icmag.com @8.8.8.8
[...]
;; ANSWER SECTION:
icmag.com. 300 IN A 188.114.96.7
icmag.com. 300 IN A 188.114.97.7
[...]IPv6
Here's a request to the IPv6 endpoint, notice the --resolve option to curl, and the Connected to line. Also, the third to last line, which shows the total time it took (20 seconds). I edited out the uninteresting bits.
Code:
$ time curl -v -o /dev/null "https://www.icmag.com/js/vendor/jquery/jquery-3.5.1.min.js?_v=9af17e8a" --resolve 'www.icmag.com:443:2a06:98c1:3121::7'
* Added www.icmag.com:443:2a06:98c1:3121::7 to DNS cache
* Hostname www.icmag.com was found in DNS cache
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Trying [2a06:98c1:3121::7]:443...
* Connected to www.icmag.com (2a06:98c1:3121::7) port 443
[...]
{ [1360 bytes data]
100 89476 100 89476 0 0 4438 0 0:00:20 0:00:20 --:--:-- 4314
* Connection #0 to host www.icmag.com left intact
real 0m20,182s
user 0m0,059s
sys 0m0,026sIPv4
The same request to the IPv4 endpoint takes only half a second to complete:
Code:
$ time curl -v -o /dev/null "https://www.icmag.com/js/vendor/jquery/jquery-3.5.1.min.js?_v=9af17e8a" --resolve 'www.icmag.com:443:188.114.96.7'
* Added www.icmag.com:443:188.114.96.7 to DNS cache
* Hostname www.icmag.com was found in DNS cache
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Trying 188.114.96.7:443...
* Connected to www.icmag.com (188.114.96.7) port 443
[...]
{ [5467 bytes data]
100 89476 100 89476 0 0 150k 0 --:--:-- --:--:-- --:--:-- 150k
* Connection #0 to host www.icmag.com left intact
real 0m0,597s
user 0m0,053s
sys 0m0,018sWorkaround
I disabled IPv6 and the website is fast and snappy. But that's not a desirable solution, obviously, so only suited for a quick test. If you're a linux user, you can disable IPv6 for an interface with (as root, and obviously change the wlp0s20f3 to the identifier of your interface)
Code:
sysctl -w net.ipv6.conf.wlp0s20f3.disable_ipv6=1The less invasive option is to add a hosts entry to your /etc/hosts, like:
Code:
188.114.97.7 www.icmag.com icmag.comI think adding a hosts entry is also possible in Windows, but I haven't touched it in 20 years, so I don't know how to do it there.
*edit* HERE is a howto for windows users.
Explanation
If you add a hosts entry, it will tell your computer to bypass the DNS lookup, and use the IP you configured to resolve requests to icmag.com, thus not seeing the IPv6 and not hitting the IPv6 endpoint. This is the least invasive option for a workaround. But note the downside: You will have to remember it, and you will have to edit it in case the IPs for icmag change. Usually this doesn't happen very often, but it might.
@admins of icmag
I'm a devops/cloud/sre freelancer and happy to assist in debugging, if necessary. Feel free to reach out. It would be nice to find a solution for this, as disabling IPv6 isn't desirable and I couldn't find an extension, that will disable either happy eyeballs or v6 DNS lookups for a certain website. So for now, a static /etc/hosts entry it is for me :/
EDIT
I have noticed this issue on different websites using cloudflare in the meanwhile as well.
Also, here's an issue report on the cloudflare forum.
Last edited:
