== Internet routing can now vary based on things you wouldn't expect [[Today Toronto had a little issue with Cloudflare https://twitter.com/thatcks/status/1375861196759400450]], which gave me a chance to learn a useful lesson about the modern Internet and how it routes traffic. The summary of the lesson is that the venerable Unix _traceroute_ command may not be your friend any more. I come from an era of a relatively simple Internet. Back then, the path that your packets took through the network was expected to depend only on the destination and the source IPs. Things in the middle might drop some traffic or filter parts of it out, but the path was the same whether you were using ICMP, UDP, or TCP, and regardless of what port TCP or UDP port you were connecting to. In this environment, _ping_ and _traceroute_ were reliable diagnostics in general; if routes weren't flapping, _traceroute_ would tell you the path that all of your traffic was using, while _ping_ could tell you that the target host was there. (If something pinged but didn't respond to the port you wanted, it was a firewall issue.) The Cloudflare issue today did not behave like that. In particular, plain _traceroute_ reported one path, a short five-hop one, while '_traceroute -T -p 443_' reported a rather different ten-hop path that seemed to take a detour off to Chicago before coming back to Toronto (and not reaching the target Cloudflare IP). At one level, port-based routing makes a certain amount of sense; it's a lower level version of application load balancers, and why go to all the bother of doing complicated things just to reject UDP packets that you don't handle. At another level it makes troubleshooting and testing more complicated, especially for outside people. ICMP, random UDP traffic, and actual TCP traffic to specific ports (or emulations of it) may go to completely different places, so information gathered in one way for one of them doesn't necessarily apply to anything else. Fortunately not everything is like this. Unfortunately the people who are most likely to be like this are the large cloud providers and [[CDNs https://en.wikipedia.org/wiki/Content_delivery_network]], and those collectively host a lot of websites and places of interest (and their complexity provides more room for subtle problems). For myself, my lesson learned from this is that if I'm trying to check out the network path to some outside place, I should use '_traceroute -T -p 443_' (or the applicable port, but HTTPS is the most likely). Once [[HTTP/3 https://en.wikipedia.org/wiki/HTTP/3]] becomes common, I'll potentially also want to check with UDP port 443 (although that gets complicated fast). Plain _ping_ and _traceroute_ are not as trustworthy as they used to be any more.