Hacker Newsnew | past | comments | ask | show | jobs | submit | LenHolgate's commentslogin

I agree.


I posted an update when HyBi 13 came out; http://www.lenholgate.com/blog/2011/09/the-websocket-protoco...

The wording was improved around the suggestion to provide only a message based API.

I think the WebSockets protocol ended up being a little more than it should have been. You have to understand that it was being pulled in all sorts of directions by the working group members and that there are good reasons for all of the parts of the protocol (though some of those parts could work better with other parts IMHO). It had to be finished at some point though and I think the working group did a good job in the end.

Personally I think it would have been better had it been explicitly stream based from a user's perspective, but then I don't have the javascript/browser background to know how foolish that probably sounds.


The argument FOR 63 bit message sizes was that you could effectively turn the message based protocol into a stream, except, unfortunately, the "stream" has a limit even if it seems plenty big enough now.

Personally I wouldn't have included the 63 bit message size.


What's the point? What's the difference between a stream of messages, and a stream of bytes? Nothing.


The difference is that with a stream of websocket messages every so often you need to deal with the framing. With a stream of bytes you don't. This precludes the use case that was the basis of the argument for 63 byte messages if you ever have to 'stream' a 'message' that needs to be longer... Sure you can send it as multiple fragments but then you can't do the 'here's a file handle, read the stream' thing that was proposed.


The protocol preserves message boundaries but not fragment boundaries. You may send a message of, say, 100 bytes and get 100 x 1 byte fragments arrive, or you may send 100 x 1 byte fragments and get 100 bytes in a single frame. The main issue, for me, at the time, was that when you get that first 1 byte frame there's no way to know how big the resulting message will be.

Luckily there's a rather excellent compliance test suite, here: http://www.tavendo.de/autobahn/testsuite.html which should go a long way to help nail interop issues.


when you get that first 1 byte frame there's no way to know how big the resulting message will be

So that's a design tradeoff. I've implemented protocols that did it both ways and it's definitely easier on the guy trying to implement a library or other receiving application if he can get a reasonable upper limit on the size of the messages.

But on the other hand, by not requiring the total message length be known in advance, it eases the logical (and memory) burden on the sender. Often the sender will be an overloaded server.

Nothing can prevent a higher-level protocol on top of WS from negotiating its own max message size.

So the design choice that was made would seem to allow optimizations for the overloaded server case without prohibiting other optimizations. This is typical of W3C protocols.


Well, all frames actually do have length in the header. It's just not the first byte... it's the lowest 7 bits of the second byte, and possibly more (check the spec).

So, if I can read 2 bytes from my buffer, I can get a good idea of the message size. If I only have 1 byte, I rewind and wait until I receive another.


But we're talking about messages, not frames. http://tools.ietf.org/html/rfc6455 "frames have no semantic meaning"

If Websocket developers begin making unwarranted assumptions about message framing and fragmentation you will regret it. BELIEVE ME.

/me gets back to bugfixing


I'm not sure I follow you to get to it being a "silly conclusion".

As I said, the draft at the time suggests presenting whole messages to the application layer. The parser can't know it has a whole message until it gets the final frame... This could lead to interesting memory usage ;)

The protocol provides for a series of infinitely long messages, each separated by a terminator. I don't have a problem with that in itself, but the draft at the time was misleading to suggest otherwise...


I've no problem with the lack of a max message size in the RFC, what could cause problems is the fact that it needs to be passed between client and server "out of band", i.e. at the application protocol layer rather than at the websocket protocol layer. Also bear in mind that this blog entry was written based on Draft HyBi 09 and not the final RFC; the wording has changed somewhat since then.

The draft in question suggested that providing a message based interface to application code was possible and that the parser could/should deliver only complete messages to the application code. That's hard to do if you also want to allow for the 'endless streaming' scenario that others on the working group were fond of. The result was a bit of a mess.

The final RFC addresses some of this, but there's no getting around the fact that the websocket protocol itself can't tell you how big a message is until you get the final frame.

Sure you can work around all of this even for a generic parser but the initial wording in the draft in question could lead you towards the wrong design if you're not careful.


As I mentioned above, I agree that that needing to pass maximum message size out of band makes some things more difficult. Whether that was the right tradeoff in terms of convenience vs protocol complexity I think has yet to be seen. At any rate, an extension to perform this in band should be trivial. Perhaps I will try writing one to test out my extension handling code.

I do see your point on the "endless streaming" section of the RFC. Stating that "(section 5.4) The primary purpose of fragmentation is to allow sending a message that is of unknown size when the message is started without having to buffer that message." implies that a web socket implementation should support this sort of operation. Indeed, if you want to support sending messages of unknown size you must expose an interface more complicated than the default message based one.

That said, a message only implementation that does not allow sending unknown sized messages is 100% compliant with both the spec and receiving such messages. The RFC probably could have made this fact more clear. I believe that endless streaming mode will not be a common use case and have not implemented it in my generic WebSocket library. I do believe, however, that fragmentation of messages provides important benefits even without unknown size sends. Once you have message fragmentation there is no additional protocol cost to allow unknown size sends.


I agree with all you're saying.

The wording of the RFC has improved since that draft and the flexibility could be useful in some scenarios.

I ended up with an API which can be asked to deliver complete messages 'if possible' given the buffers provided by the client of the API. If it's not possible and the buffer becomes full then the API simply gives you the fragment of data and tells you if it knows how much more there is to come or not.


I expect that yes, it would be useful to use RIO with Node.

What I'm hoping to find, once I've got RIO integrated into my IOCP framework, is that 'faster path' from user mode send/recv calls to/from the network stack will result in performance improvements for IOCP based servers that switch to using RIO. It likely wont be as much as if you used RIO in a tight loop polled situation, but you should still get some benefits from the pre-locked buffers and the reduced amount of work needed during send and recv calls. Going via the IOCP for completion notification will result in some kernel mode transitions but it's necessary to allow you to scale RIO to many thousands of connections.

I'll post some comparative perf tests once I've got a bit further.


That'll be great --- Thanks!

I look forward to seeing some results from you


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: