When I initially set up OCSP stapling while I was modernizing our Apache TLS configurations, I followed the standard setup from the Mozilla SSL configuration generator (as is my usual habit). For OCSP stapling, the configuration this generates was (and is) just:

SSLUseStapling On
SSLStaplingCache "shmcb:logs/ssl_stapling(32768)"

Then recently our web servers at work couldn't get a good answer from Let's Encrypt's OCSP servers, for reasons that aren't clear to us. Some people experienced issues where their Firefox would refuse to connect to these web servers, rejecting things with a SEC_ERROR_OCSP_TRY_SERVER_LATER error.

(It's possible that this actually comes from Firefox itself directly querying the LE OCSP servers. These people were inside the same networks as the web servers and our problem could well have been a firewall or network reachability issue to LE's OCSP servers, instead of an issue on LE's side.)

This experience made me look into what happens when OCSP stapling runs into errors, and also into the Apache documentation. The result of this is that I now think that you should always tell Apache to not return OCSP errors, by also adding the Apache configuration option for this:

SSLStaplingReturnResponderErrors off

(The default is 'on'.)

This does the most aggressive version of handling OCSP problems; if set to off, the documentation says 'only responses indicating a certificate status of "good" will be included in the TLS handshake'. Expired responses, any errors, and any other certificate status causes Apache to not include OCSP stapling information at all.

(You may also want to see this Apache bug.)

PS: Firefox still defaults to checking certificate status through OCSP if necessary, but you can change this if you want to. The normal preferences only let you turn this off entirely, but if you go into about:config and set security.OCSP.enabled to the value of '2', Firefox will do OCSP checks for EV certificates but not for normal ones. Given the increasing disuse of EV certificates, I don't think it's worth bothering; just turn off OCSP checking entirely.