The reason you want to use EAX or CCM instead of CTR+HMAC is that EAX and CCM are standard constructions. CTR+HMAC isn't.
Let's be clear about this, Colin. You're saying that EAX and CCM implementations are so untrustworthy that lay developers should implement their moral equivalents (substituting CBC-MAC and OMAC for HMAC) by hand.
In order to use CTR+HMAC, you have to design your own construction to do so. You likely have to implement HMAC yourself. You have to determine when to apply it, and where to put it in your messages.
CCM and EAX remove those degrees of freedom, which is exactly what you want: the fewer decisions a developer/designer needs to make, the fewer opportunities for things to blow up.
It's the places where primitives join together --- encryption with MAC, for instance --- where crypto protocols are broken. The person who uses OpenSSL is far more likely to make a fatal error than the OpenSSL developers themselves.
With all due respect to Colin, I think this take on how to minimize attack surface is, well, asinine. I think it's based on a bias against crypto library developers, based on the fact that Colin probably knows more about crypto than they do.
Stipulated.
But the point isn't that OpenSSL developers are better, or even comparable to Colin. It's that their code is 1000x more likely to be reviewed than anything anybody else writes. Your hand-hacked CTR + HMAC protocol isn't going to be reviewed by anyone who doesn't have a direct financial interest in your code.
The reason you want to use EAX or CCM instead of CTR+HMAC is that EAX and CCM are standard constructions. CTR+HMAC isn't.
I don't know from where you get the idea that Encrypt-then-MAC is not a standard construction. Encrypt-then-MAC was a standard construction long before EAX and CCM were invented -- in fact, in the paper where EAX is introduced, it is actually defined as an Encrypt-then-MAC composition!
It's the places where primitives join together --- encryption with MAC, for instance --- where crypto protocols are broken.
If you look carefully at recent breaks, they have been in Encrypt-and-MAC and MAC-then-Encrypt -- both of which I specifically mention as inferior to Encrypt-then-MAC.
With all due respect to Colin, I think this take on how to minimize attack surface is, well, asinine. I think it's based on a bias against crypto library developers, based on the fact that Colin probably knows more about crypto than they do.
I apply the same approach of minimizing attack surface even when it's only my own code which is being attacked. The question isn't how prone some code is to vulnerabilities (although in the examples I gave, there certainly is a poor track record) -- the issue is engineering code to eliminate potential risks.
I don't find your argument particularly convincing. This "encrypt-then-MAC" thing is a total red herring. You're not advocating for the idiom. You're advocating for people to build their own nonstandard versions of it, because you don't like the library code that already exists for the standard versions.
Let's cut to the chase. I can point out HMAC vulnerabilities. Nate's Google Keyczar finding. SNMPv3. That's 15 seconds worth of research. [edit SSH screwed up HMAC as well.]
It should be at least as easy for you to give me two published implementation flaws in a CCM or EAX implementation. Go.
I can point out HMAC vulnerabilities. Nate's Google Keyczar finding.
Funny you mention that. The flaw Nate pointed out in Google keyczar can also affect CCM or EAX (or, for that matter, any authenticated encryption method).
It should be at least as easy for you to give me two published implementation flaws in a CCM or EAX implementation. Go.
Of course, neither of these two papers mention CCM or EAX -- but the flaws they mention apply to both CCM and EAX (because an attacker can cause you to input a block of his choice to your AES core) while not affecting CTR-then-HMAC.
First, the flaw in Keyczar didn't affect CCM (or CBC-MAC) or EAX (or OMAC).
Second, I cited SSH, SNMPv3, and Keyczar. You responded with localhost cache timing of AES and remote cache timing of naked AES. There's an obvious difference between the two sets: one consists of real attacks, the other of speculative attacks. If your point is valid, it shouldn't be hard for you to name one system in which CCM or EAX were "broken" --- and I'll give you any definition of broken you choose --- because of these papers.
They're both great papers. But I don't think they make the argument you think they do.
First, the flaw in Keyczar didn't affect CCM (or CBC-MAC) or EAX (or OMAC).
Of course not -- keyczar doesn't implement CCM or EAX. But the fact that CCM and EAX are obscure shouldn't count as a point in their favour.
There's an obvious difference between the two sets: one consists of real attacks, the other of speculative attacks.
The Bernstein and Oskiv-Shamir-Tromer attacks were not at all speculative. They showed the concrete theft of a key.
If your point is valid, it shouldn't be hard for you to name one system in which CCM or EAX were "broken" --- and I'll give you any definition of broken you choose --- because of these papers.
I don't know of any systems which use CCM or EAX in software on general-purpose hardware -- but if you name me a system which uses OpenSSL's AES code circa early 2005 in EAX mode, I'll name you a system which was vulnerable to a timing side channel.
Tally ho, Colin! Looking forward to what you find; I'm sure I'll learn something. If Bernstein's attack was really that relevant to real-world cryptosystems, I'm sure you'll come back with something fun.
The approach also increases the chicken-and-egg problem: if EAX would be more widely used, it would be safer, but since it isn't widely used, it might not be as safe and thus should not be used, etc... The only way to break that cycle is by starting to use EAX.
By repeatedly reinventing your own CTR+HMAC construction, and building the code to implement it yourself, you are in fact locking yourself in a permanent state of immaturity.
Which is my real issue. I think Colin's probably right that --- in the abstract --- using HMAC (which doesn't use AES) instead of OMAC (which does) addresses an attack vector. But that attack vector is extremely unlikely, and it's OpenSSL's problem, not yours. Building your own protocol adds tens more vectors, all of which are your problem, and all of which are more likely than a key extraction attack through OMAC that will make your attacker famous.
It'd be awesome if someone with great visual communication skills (Amy Hoy-/Dustin Curtis-/Bret Victor-style) teamed up with cperciva's intense summaries of encryption knowledge. That would be a helluva one-two punch.
That's an interesting idea, but I'm not sure how well it would work in practice. Maybe I'm weird, but I find that making things look nice (whether via improved formatting or the addition of pictures) helps to draw readers' attention, it doesn't actually help get any more information across.
Merely making things look nice won't help, but sometimes things like dataflow diagrams can convey information more quickly and clearly than just words — maybe especially to people who are particularly visually oriented.
ECB, CBC, CTR, CFB, and OFB are all "XORable", although if you're pointing out that CTR ciphertexts that share a keystream can be XOR'd to decrypt streams, that's a valid point, and one that a MAC (of any type) doesn't solve.
You're so far better off not trying to implement any of this stuff that it's almost not worth studying. PGP at rest, TLS in motion. Spend your time and heartache on something that will make you money, not something that will get you laughed at by asshole pentesters.
There's the problem with keystream reuse, yes, and also the problem shared to some degree by CBC and OFB (but not ECB) that's at its worst in CTR — that you can flip single bits in the plaintext by flipping arbitrary bits in the ciphertext. (But of course that's why you use a correctly-implemented MAC, right?)
I concur with you that new cryptographic protocols are likely to be flawed, and your recent blog post on that subject was excellent reading.
Yes, I suppose that's a unique implementation risk for CTR and in that case Kragen's concerns about XOR are valid as well since you've got a stream cipher with a short period.
I was more thinking about how the cipher input has a predictable structure, in particular every pair of adjacent blocks differ by only a single fixed bit. This isn't an issue unless the underlying block cipher is badly broken, but it still feels unnecessarily risky.
OFB has the same problem with IV reuse, but because CTR divides the IV into a nonce and a counter, it basically begs you to make the error by giving you two opportunities to make it.
Attackers get to arbitrarily change the ciphertext, and can watch the recipient carefully to see how it reacts to different altered ciphertexts. With a bad AES implementation, things like timing differences can leak key bits.
That's not much of a problem with CTR mode --- CTR encrypts keystream, and XORs your data, so your data doesn't actually influence AES state much --- but it's a potential problem with CBC-MAC or OMAC, which do feed your data through AES.
The issue is that the fastest MACs use AES themselves, usually with the same key as was used to encrypt. CBC-MAC, OMAC, GCM, Poly1305-AES, all generate a MAC block (128 bytes, which you can truncate to whatever) in part using AES with your encrypting key.
It feels to me like Colin has a legitimate point --- side channel key extraction would be pretty hard if you had to do it from HMAC instead of with AES --- but that the person who actually came up with a side-channel exploit in a real-world OMAC (EAX) system would get famous over it.
The person who breaks your hand-hacked CTR+HMAC code, which will inevitably contain flaws because it will get no real review... that person doesn't get famous. Her win was entirely predictable.
The issue isn't that EAX is easier to implement than CTR+HMAC. EAX is actually harder to implement than CTR+HMAC.
The issue is that someone will have already written EAX for you, because EAX is a NIST standard that is used in actual fielded systems. There is no NIST standard AE construction (that I know of; I'm not an expert) that combines CTR and HMAC. You'll have to implement it yourself.
My "actual citation of a Rivest student coding for Google making the HMAC error" argument has a THAC0 of 5, and your "yes, but" argument has a THAC0 of 15. You might win this debate, but I doubt it.
Colin points out (rightly, in the narrow scope he drew) that Google could have made the same error with CBC-MAC or OMAC; they'd still be comparing bytes directly, not doing the safe XOR comparison.
But Colin is making my point for me. Because if you used EAX or CCM, you wouldn't be implementing that compare for yourself; it'd be taken care for you by the block cipher mode, which would yield either a plaintext or a failure.
Yes, the library designer could make that mistake too; after all, Google did! But again, that's my point. Google making this error was a newsworthy event, and the mere fact Google implemented it caused Nate Lawson to review the code and find it. You have none of those benefits. You're going to make the same error, and nobody's going to find it until it matters that you've not made it.
Okay, but using an encryption library doesn't guarantee your code is free of mistakes either. Just initiating a generic cryptography library requires a certain amount of specialised knowledge. You have to get the right mode, create a secure key, know how to correctly use an IV or whatever additional parameters the particular encryption algorithm needs.
Compared to all that, comparing two HMACs seems relatively straightforward.
I don't see how having a design goal to allow for a key reuse explains the "usually" part of "fastest MACs use AES themselves, usually with the same key as was used to encrypt". What you listed was the list of MACs that, from my experience, are meant to be used with their own key. That's what I asked about - where did you see them used sharing the key with an encryption cycle ?
I'm not sure if we're splitting hairs or just talking past each other, but can we figure out why it is we care about the world "usually" versus "sometimes" or whatever it is you think is the correct modifier? What's the meta-issue here?
I fully agree on the cypher then MAC policy. Deciphering is time consuming and by feeding bogus data could be used in a DOS attack. Without prior checking it may be possible to indirectly probe the deciphering process.
What I am not so sure is the choice of CTR. I currently prefer CFB.
CTR cyphers a value that changes with only a few bits, while CFB uses previous output which is much more random, though exposed, since it is the ciphered text.
Security under those circumstances is a design requirement for AES. CTR is secure if AES is secure. If known plaintext pairs differing in only a few bits break AES, we have bigger problems than CTR mode.
I think I'm missing something really basic here. If you encrypt-then-mac, what is to prevent an attacker from appending a valid MAC to a forged ciphertext (in order to mount a DOS attack for example)?
Let's be clear about this, Colin. You're saying that EAX and CCM implementations are so untrustworthy that lay developers should implement their moral equivalents (substituting CBC-MAC and OMAC for HMAC) by hand.
In order to use CTR+HMAC, you have to design your own construction to do so. You likely have to implement HMAC yourself. You have to determine when to apply it, and where to put it in your messages.
CCM and EAX remove those degrees of freedom, which is exactly what you want: the fewer decisions a developer/designer needs to make, the fewer opportunities for things to blow up.
It's the places where primitives join together --- encryption with MAC, for instance --- where crypto protocols are broken. The person who uses OpenSSL is far more likely to make a fatal error than the OpenSSL developers themselves.
With all due respect to Colin, I think this take on how to minimize attack surface is, well, asinine. I think it's based on a bias against crypto library developers, based on the fact that Colin probably knows more about crypto than they do.
Stipulated.
But the point isn't that OpenSSL developers are better, or even comparable to Colin. It's that their code is 1000x more likely to be reviewed than anything anybody else writes. Your hand-hacked CTR + HMAC protocol isn't going to be reviewed by anyone who doesn't have a direct financial interest in your code.