cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Doubleday (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4065) Bogus MemoryMeter liveRatio calculations
Date Tue, 20 Mar 2012 21:57:37 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233847#comment-13233847
] 

Daniel Doubleday commented on CASSANDRA-4065:
---------------------------------------------

bq. That's probably possible in theory, but there isn't a fixed-size overhead for structures
like CLHM.

Well yes - I though of an approximation.

But maybe even easier (still conservative but able to heal):

{noformat}
if (newRatio > cfs.liveRatio) {
    cfs.liveRatio = newRatio;
} else {
    cfs.liveRatio = (cfs.liveRatio + newRatio) / 2.0;
}
{noformat}

 
                
> Bogus MemoryMeter liveRatio calculations
> ----------------------------------------
>
>                 Key: CASSANDRA-4065
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4065
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.8
>            Reporter: Daniel Doubleday
>            Priority: Minor
>
> I get strange cfs.liveRatios.
> A couple of mem meter runs seem to calculate bogus results: 
> {noformat}
> Tue 09:14:48 dd@blnrzh045:~$ grep 'setting live ratio to maximum of 64 instead of' /var/log/cassandra/system.log
>  WARN [MemoryMeter:1] 2012-03-20 08:08:07,253 Memtable.java (line 193) setting live ratio
to maximum of 64 instead of Infinity
>  WARN [MemoryMeter:1] 2012-03-20 08:08:09,160 Memtable.java (line 193) setting live ratio
to maximum of 64 instead of Infinity
>  WARN [MemoryMeter:1] 2012-03-20 08:08:13,274 Memtable.java (line 193) setting live ratio
to maximum of 64 instead of Infinity
>  WARN [MemoryMeter:1] 2012-03-20 08:08:22,032 Memtable.java (line 193) setting live ratio
to maximum of 64 instead of Infinity
>  WARN [MemoryMeter:1] 2012-03-20 08:12:41,057 Memtable.java (line 193) setting live ratio
to maximum of 64 instead of 67.11787351054079
>  WARN [MemoryMeter:1] 2012-03-20 08:13:50,877 Memtable.java (line 193) setting live ratio
to maximum of 64 instead of 112.58547951925435
>  WARN [MemoryMeter:1] 2012-03-20 08:15:29,021 Memtable.java (line 193) setting live ratio
to maximum of 64 instead of 193.36945063589877
>  WARN [MemoryMeter:1] 2012-03-20 08:17:50,716 Memtable.java (line 193) setting live ratio
to maximum of 64 instead of 348.45008340969434
> {noformat}
> Because meter runs never decrease liveRatio in Memtable (Which seems strange to me. If
past calcs should be included for any reason wouldn't averaging make more sense?):
> {noformat}
> cfs.liveRatio = Math.max(cfs.liveRatio, newRatio);
> {noformat}
> Memtables are flushed every couple of secs:
> {noformat}
> ColumnFamilyStore.java (line 712) Enqueuing flush of Memtable-BlobStore@935814661(1874540/149963200
serialized/live bytes, 202 ops)
> {noformat}
> Even though a saner liveRatio has been calculated after the bogus runs:
> {noformat}
> INFO [MemoryMeter:1] 2012-03-20 08:19:55,934 Memtable.java (line 198) CFS(Keyspace='SmeetBlob',
ColumnFamily='BlobStore') 
>    liveRatio is 64.0 (just-counted was 2.97165811895841).  calculation took 124ms for
58 columns
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message