incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: A few questions on row caching and read consistency ONE
Date Thu, 18 Aug 2011 15:27:16 GMT
On Thu, Aug 18, 2011 at 10:36 AM, Stephen Henderson <
stephen.henderson@cognitivematch.com> wrote:

> Thanks Ed/Aaron, that really helped a lot.
>
>
>
> Just to clarify on the question of writes (sorry, I worded that badly) - do
> write operations insert rows into the cache on all nodes in the replica set
> or does the cache only get populated on reads?
>
>
>
> Aaron – in terms of scale, our ultimate goal is to achieve 99% reads under
> 5ms (ideally <1ms) with upto 20,000 operations a second (split 60/40
> read/write) and upto 2 billion keys. That’s the 12-18 month plan at least,
> short-term we’ll be more like 1000 ops/sec and 10 million keys which I think
> cassandra could cope with comfortably. We’re currently working out what the
> row-size will be, but hoping to be under 2kb max.  Consistency isn’t
> massively important. Our use case is as a user-profile store for serving
> optimised advert-content with quite tight restrictions on response time, so
> we have say 10ms to gather as much data about a user as possible before we
> have to make a decision on which creative to serve. If we can read a profile
> from the store in this time we can serve a personalised ad with a higher
> chance of engagement so low-latency is key requirement.
>
>
>
> Edward – thanks for the link to the presentation slides. A bit off-topic,
> but have you ever had a look at CouchBase (previously “membase”)? It’s
> basically memcached with persistence, fault-tolerance and online scaling.
> It’s the main alternative platform we’re considering for this project and on
> paper it sounds perfect, though we have a few concerns about it (mainly lack
> of active community, another nosql platform to learn and general uncertainty
> over the upcoming 2.0 release). We’re hoping to do some stress test
> comparison tests between the two in the near future and I’ll try to post the
> results if they’re not too company-specific.
>
>
>
> Thanks again,
>
> Stephen
>
>
>
> *From:* Edward Capriolo [mailto:edlinuxguru@gmail.com]
> *Sent:* 18 August 2011 14:14
> *To:* user@cassandra.apache.org
> *Subject:* Re: A few questions on row caching and read consistency ONE
>
>
>
>
>
> On Thu, Aug 18, 2011 at 5:01 AM, Stephen Henderson <
> stephen.henderson@cognitivematch.com> wrote:
>
> Hi,
>
> We're currently in the planning stage of a new project which needs a low
> latency, persistent key/value store with a roughly 60:40 read/write split.
> We're trying to establish if Cassandra is a good fit for this and in
> particular what the hardware requirements would be to have the majority of
> rows cached in memory (other nosql platforms like Couchbase/Membase seem
> like a more natural fit but we're already reasonably familiar with cassandra
> and would rather stick with what we know if it can work).
>
> If anyone could help answer/clarify the following questions it would be a
> great help (all assume that row-caching is enabled for the column family).
>
> Q. If we're using read consistency ONE does the read request get sent to
> all nodes in the replica set and the first to reply is returned (i.e. all
> replica nodes will then have that row in their cache), OR does the request
> only get sent to a single node in the replica set? If it's the latter would
> the same node generally be used for all requests to the same key or would it
> always be a random node in the replica set? (i.e. if we have multiple reads
> for one key in quick succession would this entail potentially multiple disk
> lookups until all nodes in the set have been hit?).
>
> Q. Related to the above, if only one node recieves the request would the
> client (hector in this case) know which node to send the request to directly
> or would there be potentially one extra network hop involved (client ->
> random node -> node with key).
>
> Q. Is it possible to do a warm cache load of the most recently accessed
> keys on node startup or would we have to do this with a client app?
>
> Q. With write consistency ANY is it correct that following a write request
> all nodes in the replica set will end up with that row in their cache, as
> well as on disk, once they receive the write? i.e. total cache size is
> (cache_memory_per_node * num_nodes) / num_replicas.
>
> Q. If the cluster only has a single column family, random partitioning and
> no secondary indexes, is there a good metric for estimating how much heap
> space we would need to leave aside for everything that isn't the row-cache?
> Would it be proportional to the row-cache size or fairly constant?
>
>
> Thanks,
> Stephen
>
>
> Stephen Henderson - Lead Developer (Onsite), Cognitive Match
> stephen.henderson@cognitivematch.com | http://www.cognitivematch.com
>
>
>
> I did a small presentation on this topic a while back.
> http://www.edwardcapriolo.com/roller/edwardcapriolo/resource/memcache.odp
>
>
> 1.
>
> a) All reads go to all replica nodes. Even those at READ.ONE. UNLESS you
> lower the read_repair_chance for the column family.
>
> b) Read could hit random nodes same node unless you confirgure dynamic
> snitch to pin the request to a single node. This is described in the
> cassandra.yaml
>
>
>
> 2. Hector and no client that I know of routes requests to proper nodes
> based on topology. No information of know of has proven this matters.
>
>
>
> 3. Cassandra allows you to save your caches so your node will start up warn
> (saving large rowcache is hard, large key cache is easy)
>
>
>
> 4. Write.ANY would not change how caching works.
>
>
>
> 5. There are some calculations out there based on size of rows. One of the
> newer features of cassandra is it automatically resizes the row cache under
> memory pressure now. You still have to feel it out but you do not have to
> worry about setting it too high as much anymore.
>
>
>
> One more note. I you have mentioned the row cache which is awesome it you
> can utilize it correctly and your use case is prefect but key cache + page
> cache can server very fast as well.
>
>
>
> Thank you,
>
> Edward
>
>
>
>
>


Wait membase is couchbase? I thought it was northscale? (I can not keep up).
It seems to have coordinators or masters.
http://www.slideshare.net/tim.lossen.de/an-introduction-to-membase
Any solution where all the read write traffic travels through a master I do
not believe to be scalable. Other solutions that use a master for
coordination election but read or write directly to nodes are "more"
scalable but more fragile.

Q. Why does every scalable architecture except Cassandra seem to have master
nodes ? :)

It is not in the YCSB so hard to say how fast it is or how well it
performs.

Mime
View raw message