It's not as bad as ASN.1. But when you need a "strict mode" for a binary format, and otherwise allow ambiguous decoding, there's something wrong. Something that probably can be exploited. The RFC admits this.
How does the density compare with JSON run through GZIP compression?
It seems rather that the spec allows, but doesn't require, lenience in what data the encoder accepts. If you want to write an encoder that always errors when given invalid data (that is always "strict"), that's fine. But if you're going to be liberal in what you accept, you are required to also implement a strict mode.
I have implemented CBOR for Python and Go. In my testing:
CBOR+GZIP is slightly smaller, about 10%, than JSON+GZIP
CBOR also parses faster, sometimes as much as 3-5x speedup.
I'm not sure what 'ambiguous decoding' you're referring to. I've seen some complains that an int might decode to an int32 or int64 depending on the value but be different than the original storage size. e.g. '42' stored from an int64 might unpack to an int8 depending on the language and implementation of the decoder.
How does the density compare with JSON run through GZIP compression?