OTR is definately an inspiration here. I need to add the HMAC signing and key ratcheting. When I started there was no good OTR implementation that I could find.I was also concerned with performance. I am looking to use telehash now. What is your opinion of telehash vs stock OTR?
I wasn't too familiar with telehash until you mentioned it. I read the docs and it looks like it's a pretty complete framework whereas OTR is more of a handshake to establish a secure channel. If you're only using telehash to secure a channel it seems like overkill. If you use its peer resolution and asynchronous messaging features it seems pretty nice.
libotr is a very good implementation of OTR (in c) [1]. I don't think performance would be a big issue on modern hardware. OTR does offer future deniability in case one of the parties gets compromised. I didn't see the same guaranty in telehash (I might have missed it).
I think they are both fine platforms but I'm far from being an expert. For anyone who is interested in the documentation I was referring to: telehash [2] and OTR [3].