Wandering Thoughts archives

2024-12-30:	I'm firmly attached to a mouse and (overlapping) windows
2024-12-24:	The TLS certificate multi-file problem (for automatic updates)
2024-12-20:	Remembering to make my local changes emit log messages when they act
2024-12-17:	We have an unusual concern when we use Let's Encrypt
2024-12-11:	The long (after)life of some of our old fileserver hardware
2024-12-09:	Maybe we should explicitly schedule rebooting our fleet every so often
2024-12-07:	PCIe cards we use and have used in our servers
2024-12-03:	The modern world of server serial ports, BMCs, and IPMI Serial over LAN
2024-11-28:	My life has been improved by my quiet Prometheus alert status monitor
2024-11-22:	My new solution for quiet monitoring of our Prometheus alerts
2024-11-21:	Our Prometheus alerting problem if our central mail server isn't working
2024-11-15:	IPv6 networks do apparently get probed (and implications for address assignment)
2024-11-14:	Your options for displaying status over time in Grafana 11
2024-11-12:	Finding a good use for keep_firing_for in our Prometheus alerts
2024-11-11:	Prometheus makes it annoyingly difficult to add more information to alerts
2024-10-29:	Doing general address matching against varying address lists in Exim
2024-10-22:	Having rate-limits on failed authentication attempts is reassuring
2024-10-16:	Our various different types of Ubuntu installs
2024-10-14:	We have lots of local customizations (and how we keep track of them)
2024-10-13:	Our local changes to standard (Ubuntu) installs are easy to forget
2024-10-12:	Some thoughts on why 'inetd activation' didn't catch on
2024-09-29:	Brief notes on making Prometheus's SNMP exporter use additional SNMP MIB(s)
2024-09-27:	Brief notes on how the Prometheus SNMP exporter's configurations work
2024-09-26:	The impact of the September 2024 CUPS CVEs depends on your size
2024-09-20:	Our broad reasons for and approach to mirroring disks
2024-09-15:	Why we're interested in FreeBSD lately (and how it relates to OpenBSD here)
2024-09-14:	Getting maximum 10G Ethernet bandwidth still seems tricky
2024-09-12:	What admin access researchers have to their machines here
2024-09-11:	Rate-limiting failed SMTP authentication attempts in Exim 4.95
2024-09-08:	I should probably reboot BMCs any time they behave oddly
2024-09-04:	Using rsync to create a limited ability to write remote files
2024-08-27:	Some reasons why we mostly collect IPMI sensor data locally
2024-08-24:	JSON is usually the least bad option for machine-readable output formats
2024-08-20:	Some brief notes on '`numfmt`' from GNU Coreutils
2024-08-15:	Workarounds are often forever (unless you work to make them otherwise)
2024-08-14:	Traceroute, firewalls, and the modern Internet: a horrible realization A note to myself about using traceroute to check for port reachability
2024-08-13:	Some thoughts on OpenSSH 9.8's PerSourcePenalties feature
2024-08-12:	Uncertainties and issues in using IPMI temperature data
2024-08-10:	Allocating disk space (and all resources) is ultimately a political decision
2024-08-03:	A surprise with the temperature distribution in our machine room
2024-07-30:	On not automatically reconnecting to IPMI Serial-over-LAN consoles
2024-07-29:	Handling (or not) the serial console of our serial console server
2024-07-28:	Our slowly growing Unix monoculture
2024-07-25:	How I almost set up a recursive syslog server
2024-07-21:	Our giant login server: solving resource problems with brute force
2024-07-20:	My home wireless network and convenience versus security
2024-07-17:	SSH has become our universal (Unix) external access protocol
2024-07-11:	Brute force attackers seem to switch targets rapidly if you block them
2024-07-06:	"Out of band" network management is not trivial
2024-07-04:	Structured log formats are not really "plaintext" logs
2024-06-29:	Plaintext is not a great format for (system) logs
2024-06-27:	Is blocking outgoing traffic by default a good firewall choice now?
2024-06-22:	A Prometheus Blackbox gotcha: (UDP) DNS replies have a low size limit
2024-06-21:	The IMAP LIST command as it interacts with client prefixes in Dovecot
2024-06-15:	We don't know what's happening on our networks
2024-06-13:	Using prime numbers for our Prometheus scrape intervals
2024-06-11:	The size of our Prometheus setup as of June 2024
2024-06-09:	OpenSSH can chose (or force) the 'shell' used for a specific SSH key
2024-06-08:	Operating services versus operating an "adequate environment"
2024-05-27:	Some notes on Grafana Loki's new "structured metadata" (as of 3.0.x)
2024-05-26:	Flaky alerts are telling you something
2024-05-23:	There are multiple uses for metrics (and collecting metrics)
2024-05-17:	The trade-offs in not using WireGuard to talk to our cloud server
2024-05-16:	Thoughts on (not) automating the setup of our first cloud server
2024-05-11:	Where NS records show up in DNS replies depends on who you ask
2024-05-08:	All configuration files should support some form of file inclusion
2024-05-07:	Some thoughts on when you can and can't lower OpenSSH's 'LoginGraceTime'
2024-05-06:	What affects what server host key types OpenSSH will offer to you
2024-05-05:	OpenSSH sshd's 'MaxStartups' setting and Internet-accessible machines
2024-05-04:	We have our first significant batch of servers that only have UEFI booting
2024-05-01:	Having a machine room can mean having things in your machine room
2024-04-28:	How I (used to) handle keeping track of how I configured software
2024-04-27:	Autoconf and configure features that people find valuable
2024-04-26:	I wish projects would reliably use their release announcements mechanisms
2024-04-24:	Pruning some things out with (GNU) `find` options
2024-04-16:	IPMI connections have privilege levels, not just IPMI users
2024-04-07:	NAT'ing on the firewall versus host routes for public IPs
2024-04-02:	An issue with Alertmanager inhibitions and resolved alerts What Prometheus Alertmanager's `group_interval` setting means
2024-04-01:	The power of being able to query your servers for unpredictable things
2024-03-30:	The Prometheus scrape interval mistake people keep making
2024-03-28:	The effects of silences (et al) in Prometheus Alertmanager
2024-03-27:	Some questions to ask about what silencing alerts means
2024-03-26:	How I would automate monitoring DNS queries in basic Prometheus
2024-03-25:	Options for diverting alerts in Prometheus
2024-03-15:	The problem of using basic Prometheus to monitor DNS query results
2024-03-14:	You might want to think about if your system serial numbers are sensitive
2024-03-11:	Why we should care about usage data for our internal services
2024-03-09:	Some thoughts on usage data for your systems and services
2024-03-01:	Options for your Grafana panels when your metrics change names
2024-02-28:	Detecting absent Prometheus metrics without knowing their labels
2024-02-27:	Our probably-typical (lack of) machine inventory situation
2024-02-22:	A recent abrupt change in Internet SSH brute force attacks against us
2024-02-09:	Compatibility lingers long after it's needed (until it gets noticed)
2024-02-05:	We might want to regularly keep track of how important each server is
2024-01-29:	What I think goes wrong periodically with our Grafana Loki on restarts Servers are (probably) starting to drop serial ports
2024-01-21:	The expected size of a gap in a Prometheus range vector (sometimes)
2024-01-20:	An example of how Prometheus's delta() function will extrapolate time ranges
2024-01-16:	What Prometheus exporters we use (as of the end of 2023)
2024-01-15:	How we monitor that our wireless network is still there in places
2024-01-09:	How far back we want our metrics to go depends on what they're for
2024-01-05:	Having a virtual machine host server has been quite useful
2024-01-03:	Ten years isn't long enough for maximum age settings
2024-01-01:	Alerting on our NTP servers having a high NTP stratum hasn't been useful

(Previous year | Next year)

By month for 2024: Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec; before 2024; after 2024.