2022-11-02
The problem of getting problem reports from (our) people
We have a somewhat unusual problem, which is that we often don't get many problem reports from our users. To be specific, not infrequently we don't get problem reports even when there are problems. Sometimes it's possible that people simply haven't noticed the problem, but other times either they don't report the problem or they don't necessarily know that something is supposed to work in the first place.
(The department has a somewhat unusual support model where other people deal directly with professors, graduate students, and so on for most issues, but even Points of Contact don't necessarily hear about problems.)
There are, of course, a host of reasons why people wouldn't report problems, as anyone who has ever encountered a problem in open source software and not filed a bug report about it can imagine. Some of the less pleasant reasons don't apply here, but we can't blame people with prior bad experiences elsewhere from worrying about it. Beyond the common reasons, I think that many of our people being graduate students, professors, postdocs, and so on creates a certain amount of cultural bias where these people expect that they're on their own for a lot of things.
(Another aspect of reporting a problem is that it's not clear if some problems are actual problems or just 'how it is'. If you find our IMAP server slow and sometimes erratic, is that just how it is or is it a sign of a problem that's worth reporting?)
Of course, from our perspective this is unfortunate, because we can't fix problems that we don't know about (and we know that in the past people have just quietly lived with problems). It's especially tricky for problems that go away after a bit or fix themselves on their own, since shorter and less frequent the problem is, the less chance any one person will be pushed into supporting it.
So, for example, I was asked in a comment on my entry on our experiences with maildir over NFS if we'd had problems with Dovecot's mbox indexing. The best answer I can give is that we haven't observed any ourselves and haven't had anything reported to us. This doesn't mean that there aren't any problems; for us, the absence of reports of problems isn't proof of lack of problems. If we want to be at all sure, we need to go look for problems, but in the system and by actually talking to people.
Sidebar: a practical example of not noticing a problem
The department's space at the university is covered by multiple wireless networks that pretty much everyone can use; there's ours, the university-wide wireless network, and Eduroam. For the most part they all let you do the same thing, and modern devices are often set up to automatically and transparently hop on to whatever wireless network they can see. The net result is that if our wireless network stops being visible somewhere but other wireless networks are still present, most people may not actually notice. Their devices would prefer our wireless network if it was there, but with it gone the devices will transparently hop to another of the available networks.
(We've had this happen, and as a result we're exploring ways to notice if our wireless network disappears somewhere. Our particular wireless environment creates unusual complexities.)