Not specific to JAB, but there's a podcast episode of Containers [0] following the end-to-end movement of coffee and the surprising amount of middlemen.
The hardest part about getting the Yubikey to work, at least under Ubuntu, is dealing with the marriage to gnome-keyring. Uninstalling it and the associated autostart gpg-agent files (see http://www.bootc.net/archives/2013/06/09/my-perfect-gnupg-ss...) seems to do the trick. The documentation for doing this is _horrendous_
[0]: https://medium.com/containers/episode-4-coffee-78ac6571caea