incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Knight <nkni...@runawaynet.com>
Subject Re: Read Latency
Date Wed, 20 Oct 2010 00:47:20 GMT
On Oct 20, 2010, at 6:30 AM, Wayne wrote:

> I am not sure how many bytes,

Then I don't think your performance numbers really mean anything substantial. Deserialization
time is inevitably going to go up with the amount of data present, so unless you know how
much data you actually have, there's no way to know if the time being taken is reasonable
or not.

> but we do convert the cassandra object that is returned in 3s into a dictionary in ~1s
and then again into a custom python object in about ~1.5s. Expectations are based on this
timing. If we can convert what thrift returns into a completely new python object in 1s why
does thrift need 3s to give it to us?


(de)serialization, even of binary protocols, is a good deal more complex than simply copying
around data you already have on the heap (or maybe even in CPU cache, if it's small enough)
in native format.

Worse, unless you're explicitly doing deep copies (which is usually the wrong thing to do),
you're probably _not_ constructing a "completely new python object". You're constructing an
object that has references to bits of data already deserialized onto the heap by Thrift. This
makes the comparison even less meaningful.

-NK
Mime
View raw message