incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yiming Sun <yiming....@gmail.com>
Subject strange row cache behavior
Date Wed, 28 Nov 2012 19:20:44 GMT
Hi,

I am trying to understand some strange behavior of cassandra row cache.  We
have a 6-node Cassandra cluster in a single data center on 2 racks, and the
neighboring nodes on the ring are from alternative racks.  Each node has
1GB row cache, with key cache disabled.   The cluster uses
PropertyFileSnitch, and the ColumnFamily I fetch from uses
NetworkTopologyStrategy, with replication factor of 2.  My client code uses
Hector to fetch a fixed set of rows from cassandra

What I don't quite understand is even after I ran the client code several
times, there are always some nodes with 0 row cache hits, despite that the
row cache from all nodes are filled and all nodes receive requests.

Which nodes have 0 hits seem to be strongly related to the following:

 - the set of row keys to fetch
 - the order of the set of row keys to fetch
 - the list of hosts passed to Hector's CassandraHostConfigurator
 - the order of the list of hosts passed to Hector

Can someone shed some lights on how exactly the row cache works and
hopefully also explain the behavior I have been seeing?  I thought if the
fixed set of the rows keys are the only thing I am fetching (each row
should be on the order of 10's of MBs, no more than 100MB), and each node
gets requests, and its row cache is filled, there's gotta be some hits.
 Apparent this is not the case.   Thanks.

cluster information:

Address         DC          Rack        Status State   Load
Effective-Ownership Token

141784319550391026443072753096570088105
x.x.x.1    DC1         r1          Up     Normal  587.46 GB
33.33%              0
x.x.x.2    DC1         r2          Up     Normal  591.21 GB
33.33%              28356863910078205288614550619314017621
x.x.x.3    DC1         r1          Up     Normal  594.97 GB
33.33%              56713727820156410577229101238628035242
x.x.x.4    DC1         r2          Up     Normal  587.15 GB
33.33%              85070591730234615865843651857942052863
x.x.x.5    DC1         r1          Up     Normal  590.26 GB
33.33%              113427455640312821154458202477256070484
x.x.x.6    DC1         r2          Up     Normal  583.21 GB
33.33%              141784319550391026443072753096570088105


[user@node]$ ./checkinfo.sh
*************** x.x.x.4
Token            : 85070591730234615865843651857942052863
Gossip active    : true
Thrift active    : true
Load             : 587.15 GB
Generation No    : 1354074048
Uptime (seconds) : 36957
Heap Memory (MB) : 2027.29 / 3948.00
Data Center      : DC1
Rack             : r2
Exceptions       : 0

Key Cache        : size 0 (bytes), capacity 0 (bytes), 0 hits, 0 requests,
NaN recent hit rate, 14400 save period in seconds
Row Cache        : size 1072651974 (bytes), capacity 1073741824 (bytes), 0
hits, 2576 requests, NaN recent hit rate, 0 save period in seconds

*************** x.x.x.6
Token            : 141784319550391026443072753096570088105
Gossip active    : true
Thrift active    : true
Load             : 583.21 GB
Generation No    : 1354074461
Uptime (seconds) : 36535
Heap Memory (MB) : 828.71 / 3948.00
Data Center      : DC1
Rack             : r2
Exceptions       : 0

Key Cache        : size 0 (bytes), capacity 0 (bytes), 0 hits, 0 requests,
NaN recent hit rate, 14400 save period in seconds
Row Cache        : size 1072602906 (bytes), capacity 1073741824 (bytes), 0
hits, 3194 requests, NaN recent hit rate, 0 save period in seconds

Mime
View raw message