cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pierre Chalamet" <>
Subject RE: CQL3 Frame Length
Date Mon, 21 Jan 2013 23:05:39 GMT
That's not a good reason imho. This would have been better to have chunks of
data (like in the good old IFF file format).
If the client is not able to read the chunk, just skip it. And frankly,
that's not a few more bytes that would have killed us.
As an example, request tracing was added pretty late and then, additional
data just landed not at the end as could have been anticipated, but before
the body of the frame. This could have been handled transparently with a
chunk format. And OK this was in rc2 and not in 1.2 so no regression
officially introduced. 
But well, it's v1 - there are still 0x7E more versions to get it better.
- Pierre
From: Theo Hultberg [] 
Sent: Saturday, January 19, 2013 6:33 PM
Subject: Re: CQL3 Frame Length

Another reason for keeping the frame length in the header is that newer
versions can add fields to frames without older clients breaking. For
example a minor release can add some more content to an existing frame
without older clients breaking. If clients didn't know the full frame length
(and were required by the specification to consume all the bytes) there
would be trailing garbage which would most likely crash the client.

> Hey Sylvain,>

> Thanks for explaining the rationale. When you look at from the perspective
> of the use cases you mention, it makes sense to be able to supply the
> reader with the frame size up front.>

> I've opted to go for serializing the frame into a buffer. Although this
> could materialize an arbitrarily large amount of memory, ultimately the
> driving application has control of the degree to which this can occur, so
> in the grander scheme of things, you can still maintain streaming

> Thanks for the heads up.>

> Cheers,>

> Ben>

> On Tue, Jan 8, 2013 at 4:08 PM, Sylvain Lebresne <

>> Mostly this is because having the frame length is convenient to have in
>> practice.
>> Without pretending that there is only one way to write a server, it is
>> common
>> to separate the phase "read a frame from the network" from the phase
>> "decode
>> the frame" which is often simpler if you can read the frame upfront.
>> if
>> you don't have the frame size, it means you need to decode the whole
>> before being able to decode the next one, and so you can't parallelize
>> decoding.
>> It is true however that it means for the write side that you need to
>> either be
>> able to either pre-compute the frame body size or to serialize it in
>> first. That's a trade of for making it easier on the read side. But if
>> want
>> my opinion, on the write side too it's probably worth parallelizing the
>> message
>> encoding (which require you encode it in memory first) since it's an
>> asynchronous protocol and so there will likely be multiple writer
>> simultaneously.
>> --
>> Sylvain
>> On Tue, Jan 8, 2013 at 12:48 PM, Ben Hood < <>> wrote:
>>> Hi,
>>> I've read the CQL wire specification and naively, I can't see how the
>>> frame length length header is used.
>>> To me, it looks like on the read side, you know which type of structures
>>> to expect based on the opcode and each structure is TLV encoded.
>>> On the write side, you need to encode TLV structures as well, but you
>>> don't know the overall frame length until you've encoded it. So it would
>>> seem that you either need to pre-calculate the cumulative TLV size
>>> you serialize the frame body, or you serialize the frame body to a
>>> which you can then get the size of and then write to the socket, after
>>> having first written the count out.
>>> Is there potentially an implicit assumption that the reader will want to
>>> pre-buffer the entire frame before decoding it?
>>> Cheers,
>>> Ben

View raw message