Wandering Thoughts archives

2017-06-15

Why I am not installing your app on my phone

For reasons beyond the scope of this entry, I spent a decent chunk of time today using my phone to amuse myself. Part of that time was reading Twitter, and part of that reading involved following links to interesting articles on various places. Quite a number of those places wanted me to install their iPhone app instead of reading things on their website, and some of them were quite obnoxious about it. For example, Medium sticks a prominent non-dismissable button in the middle of the bottom of the screen, effectively shrinking an already-too-small screen that much further.

Did I install any of the apps that these websites wanted me to? Of course not. This is not because my phone has only limited space, and it's not really because I prefer keeping my phone uncluttered. There's a much more fundamental reason: I don't trust your app. In fact, I assume that almost all apps that websites want me to use instead of reading the site are actually trojan horses.

By this, I don't mean that I expect any of these apps to quietly attack the security of my phone and attempt to compromise it (although I wouldn't take that bet on Android). I don't even necessarily expect all of these apps to demand intrusive device permissions, like constant access to location services (although I suspect that a lot of them will at least ask for lots of permissions, because maybe I'll be foolish enough to agree). I do definitely expect that all of these apps will put their app nature to 'good' use in order to spy on, track, and monetize my in-app activity to a much larger extent than their websites can. Any potential improvement in my user experience over just reading the website is incidental to their actual reason for existing, which is why they're trojan horses.

So no, I will not install your app, because I trust Apple and Safari to do a lot more to preserve my privacy against your website and your spying on me than I trust you. Safari allows various sorts of (ad)blockers, it allows me to totally discard cookies and other tracking identification that websites have foisted off on me, and in general I trust Apple to be pretty carefully restrictive of what it allows websites to do with JavaScript and what information they can get. Your app? I have no such trust; in fact I expect the exact reverse of trust.

There is nothing particularly surprising here, of course. This is simply the inevitable result of the business model of these websites. I'm not their customer, although they may pretend otherwise; instead, I am part of the product, to be packed up and sold off to advertisers. Trying to get me to accept the app is part of fattening me up for their actual customers.

(This is a bit of a grumpy rant, because I got sick and tired of all the 'install our app, really' badgering from various places, especially when it makes their websites less usable. Some of the time these nags encouraged me to close the page as not sufficiently fascinating, which may or may not have been a win for the websites in question.)

WhyNotInstallingYourApp written at 01:05:50; Add Comment

2017-06-06

The IPv6 address lookup problem (and brute force solution)

In Julia Evans' article Async IO on Linux: select, poll, and epoll, she mentioned in passing that she straceed a Go program making a HTTP request and noticed something odd:

Then [the Go program] makes 2 DNS queries for example.com (why 2? I don’t know!), and uses epoll_wait to wait for replies [...]

It turns out that this is all due to IPv6 (and the DNS standards), and it (probably) happens in more than Go programs (although I haven't straced anything else to be sure). So let's start with the problem.

Suppose that you have a 'dual-stack' machine, one with both IPv4 and IPv6 connectivity. You need to talk to a wide variety of other hosts; some of them are available over IPv4 only, some of them are available over IPv6 only, and some of them are available over both (in which case you traditionally want to use IPv6 instead of IPv4). How do you look up their addresses using DNS?

DNS currently has no way for a client to say 'give me whatever IPv4 and IPv6 addresses a host may have'. Instead you have to ask specifically for either IPv4 addresses (with a DNS A record query) or IPv6 addresses (with a DNS AAAA record query). The straightforward way for a dual-stack machine to find the IP addresses of a remote host would be to issue an AAAA query to get any IPv6 addresses, wait for it to complete (or error out or time out), and then issue an A query for IPv4 addresses if necessary. However, there are a lot of machines that have no IPv6 addresses, so a lot of the time you'd be adding the latency of an extra DNS query to your IP address lookups. Extra latency (and slower connections) doesn't make people happy, and DNS queries are not necessarily the fastest thing in the world in the first place for various reasons.

(Plus, there are probably some DNS servers and overall DNS systems that will simply time out for IPv6 AAAA queries instead of promptly giving you a 'no' answer. Waiting for a timeout adds substantial amounts of time. Properly operated DNS systems shouldn't do this, but there are plenty of DNS systems that don't operate properly.)

To deal with this, modern clients increasingly opt to send out their A and AAAA DNS queries in parallel. This is what Go is doing here and in general (in its all-Go resolver, which is what the Go runtime tries to use), although it's hard to see it in the net package's source code until you dig quite far down. Go waits for both queries to complete, but there are probably some languages, libraries, and environments that immediately start a connection attempt when they get an answer back, without waiting for the other protocol's query to finish too.

(There is a related algorithm called Happy Eyeballs which is about trying to make IPv6 and IPv4 connections in parallel and using whichever completes first. And there is a complicated RFC on how you should select a destination address out of the collection that you may get from your AAAA and A DNS queries.)

Sidebar: DNS's lack of an 'all types of IP address' query type

I don't know for sure why DNS doesn't have a standard query type for 'give me all IP addresses, either IPv4 or IPv6'. Per Wikipedia, DNS itself was created in the mid 1980s, well before IPv6 was designed. However, IPv6 itself is decades old at this point, which is lots of time to add such a query type to DNS and have people adopt it (although it might still not be universally supported, which would leave you falling back to explicit A queries at least). My best guess for why such a query type was never added is a combination of backwards compatibility worries (since initially not many DNS servers would support it, so clients would mostly be making an extra DNS query for nothing) and a general belief on the part of IPv6 people that IPv4 was going to just go away entirely any day soon, really.

(We know how that one turned out. It's 2017, and IPv4 only hosts and networks remain very significant.)

IPv6AddressLookupProblem written at 00:27:36; Add Comment

2017-06-02

My views on the JSON Feed syndication feed format

When I first read the JSON Feed version 1 specification, I came away feeling frustrated (and expressed it on Twitter) because my initial impression was that the JSON Feed people had not bothered to look at prior art and (painful) prior experiences. Then I read more, including things like Mapping RSS and Atom to JSON Feed, which made it clear that several things that I thought might be accidental omissions were in fact deliberate decisions. Now my current dominant feeling about JSON Feed is quiet sadness.

On a straightforward level I think that the current JSON Feed specification makes some bad suggestions about id elements (and also). I also think that the specification is at least loosely written overall, with imprecise language and important general qualifications that are mentioned only in one spot. I think that this is a bad idea given how I expect JSON Feed's specification to be read. Since people implementing JSON Feed seem to currently be coordinating with each other, JSON Feed may still avoid potential misunderstandings and being redefined by implementations.

Stepping beyond issues of how the specification is written, I'm sad that JSON Feed has chosen to drop various things that Atom allows. The thing that specifically matters to me is HTML in feed entry titles, because I use that quite frequently, usually for fonts. Resources like Mapping RSS and Atom to JSON Feed make it plain that this was a deliberate choice in creating the specification. I think that Atom encapsulates a lot of wisdom about what's important and useful in a syndication feed format and it would clearly be useful to have a JSON mapping of that, but that's not what JSON Feed is; it has deliberately chosen to be less than Atom, eliminating some features and some requirements outright.

(The whole thing leaves me with the feeling that JSON Feed is mostly crafted to be the minimum thing that more or less works, both in the actual content of the specification and how it's written. Some people will undoubtedly consider this praise for JSON Feed.)

As you might suspect from this, I have no plans to add JSON Feed generation to DWiki, the wacky Python-based wiki engine behind Wandering Thoughts. Among other issues, DWiki is simply not written in a way that would make generating JSON natively at all an easy process. Adding a JSON Feed is probably reasonably easy in most environments where you assemble your syndication feed as a complete data structure in memory and then serialize it in various formats, because JSON is just another format there (and these days, probably an easy one to serialize to). But for better or worse, DWiki uses a fundamentally different way of building feeds.

Should you provide a JSON Feed version of your syndication feed? I have no opinion either way. Do it if you want to, especially if it's easy. I do hope very much that we don't start seeing things that are JSON-Feed-only, because of course there are a lot of syndication feed consumers out there that certainly don't understand JSON Feed now and may never be updated to understand it.

(But then, maybe syndication feeds are on the way out in general. Certainly there has been rumbles about that in the past, although you couldn't prove it from my Atom feed fetch rates.)

JSONFeedMyViews written at 00:21:47; Add Comment

By day for June 2017: 2 6 15; before June; after June.

Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.