Let's Encrypt (really ACME) has a decent reason for (still) using CSRs
As I found out a while back (in this entry), the ACME protocol that Let's Encrypt invented and used submits its actual requests for TLS certificates as Certificate Signing Requests (CSRs), despite CSRs being famously complicated things that are theoretically full of information that Let's Encrypt and ACME don't care about. The story I heard was that this was initially done because Let's Encrypt worried that the Certificate Authority Baseline Requirements might require CSRs to properly issue a TLS certificate, but recently Matthew McPherrin shared a different and great reason on the Fediverse:
@cks We don't believe (anymore) that CSRs are required, but the biggest reason is for compatibility with existing systems. Like some security cameras will give me a CSR for their web UI, as it's the defacto format for public keys to request certs.
Before reading McPherrin's post, it hadn't occurred to me that an ACME client could submit an externally generated CSR to Let's Encrypt (or anyone else supporting ACME), but of course this is perfectly allowed. Since you can submit externally generated CSRs, the ACME protocol can be used to get a certificate for anything that can generate a CSR, including self-contained black boxes that generate keys internally and never expose them to you. As McPherrin notes, CSRs are the de facto format to use for this sort of thing, simply because so many CAs spent so long requiring you to create and submit CSRs.
(Whether any particular ACME client will support this is another issue entirely, and your mileage will vary. Plus, there are protocol issues involved. In a quick check, Certbot seems to support supplying your own CSR with the '--csr' switch to 'certbot certonly'.)
One additional thing that may want to work with CSRs is hardware security keys (generally, HSMs, but most places will probably not have full scale HSMs). Since CSRs are a de facto standard for getting CA-signed keys, the software involved may want to generate them, and certainly they won't give you the private key (that's the whole point) so even without a CSR you'd have to be able to work with the public key alone.
(With that said, many HSMs will let you generate a keypair externally and then import it. History has suggested that this may be more secure in practice.)