cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Radim Kolar (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3497) BloomFilter FP ratio should be configurable or size-restricted some other way
Date Sat, 24 Dec 2011 19:40:31 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13175760#comment-13175760
] 

Radim Kolar commented on CASSANDRA-3497:
----------------------------------------

i compiled jars with this patch and cassandra do not boots an existing node

 Opening /var/lib/cassandra/data/system/Migrations-hc-109 (757635 bytes)
 INFO [SSTableBatchOpen:1] 2011-12-24 18:26:47,326 SSTableReader.java (line 134) Opening /var/lib/cassandra/data/system/LocationInfo-hc-273
(647 bytes)
 INFO [SSTableBatchOpen:1] 2011-12-24 18:26:47,338 SSTableReader.java (line 134) Opening /var/lib/cassandra/data/system/HintsColumnFamily-hc-1
(275 bytes)
 INFO [SSTableBatchOpen:2] 2011-12-24 18:26:47,338 SSTableReader.java (line 134) Opening /var/lib/cassandra/data/system/HintsColumnFamily-hc-2
(85 bytes)
 INFO [main] 2011-12-24 18:26:47,396 DatabaseDescriptor.java (line 501) Loading schema version
ad8d50b0-2cc3-11e1-0000-b1504fb874be
ERROR [main] 2011-12-24 18:26:47,555 AbstractCassandraDaemon.java (line 372) Exception encountered
during startup
org.apache.avro.AvroTypeException: Found {"type":"record","name":"CfDef","namespace":"org.apache.cassandra.db.migration.avro","fields":[{"name":"keyspace","type":"string"},{"name":"name","type":"string"},{"name":"column_type","type":["string","null"]},{"name":"comparator_type","type":["string","null"]},{"name":"subcomparator_type","type":["string","null"]},{"name":"comment","type":["string","null"]},{"name":"row_cache_size","type":["double","null"]},{"name":"key_cache_size","type":["double","null"]},{"name":"read_repair_chance","type":["double","null"]},{"name":"replicate_on_write","type":"boolean","default":false},{"name":"gc_grace_seconds","type":["int","null"]},{"name":"default_validation_class","type":["null","string"],"default":null},{"name":"key_validation_class","type":["null","string"],"default":null},{"name":"min_compaction_threshold","type":["null","int"],"default":null},{"name":"max_compaction_threshold","type":["null","int"],"default":null},{"name":"row_cache_save_period_in_seconds","type":["int","null"],"default":0},{"name":"key_cache_save_period_in_seconds","type":["int","null"],"default":3600},{"name":"row_cache_keys_to_save","type":["null","int"],"default":null},{"name":"merge_shards_chance","type":["null","double"],"default":null},{"name":"id","type":["int","null"]},{"name":"column_metadata","type":[{"type":"array","items":{"type":"record","name":"ColumnDef","fields":[{"name":"name","type":"bytes"},{"name":"validation_class","type":"string"},{"name":"index_type","type":[{"type":"enum","name":"IndexType","symbols":["KEYS","CUSTOM"],"aliases":["org.apache.cassandra.config.avro.IndexType"]},"null"]},{"name":"index_name","type":["string","null"]},{"name":"index_options","type":["null",{"type":"map","values":"string"}],"default":null}]}},"null"]},{"name":"row_cache_provider","type":["string","null"],"default":"org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider"},{"name":"key_alias","type":["null","bytes"],"default":null},{"name":"compaction_strategy","type":["null","string"],"default":null},{"name":"compaction_strategy_options","type":["null",{"type":"map","values":"string"}],"default":null},{"name":"compression_options","type":["null",{"type":"map","values":"string"}],"default":null}]},
expecting {"type":"record","name":"CfDef","namespace":"org.apache.cassandra.db.migration.avro","fields":[{"name":"keyspace","type":"string"},{"name":"name","type":"string"},{"name":"column_type","type":["string","null"]},{"name":"comparator_type","type":["string","null"]},{"name":"subcomparator_type","type":["string","null"]},{"name":"comment","type":["string","null"]},{"name":"row_cache_size","type":["double","null"]},{"name":"key_cache_size","type":["double","null"]},{"name":"read_repair_chance","type":["double","null"]},{"name":"replicate_on_write","type":"boolean","default":false},{"name":"gc_grace_seconds","type":["int","null"]},{"name":"default_validation_class","type":["null","string"],"default":null},{"name":"key_validation_class","type":["null","string"],"default":null},{"name":"min_compaction_threshold","type":["null","int"],"default":null},{"name":"max_compaction_threshold","type":["null","int"],"default":null},{"name":"row_cache_save_period_in_seconds","type":["int","null"],"default":0},{"name":"key_cache_save_period_in_seconds","type":["int","null"],"default":3600},{"name":"row_cache_keys_to_save","type":["null","int"],"default":null},{"name":"merge_shards_chance","type":["null","double"],"default":null},{"name":"id","type":["int","null"]},{"name":"column_metadata","type":[{"type":"array","items":{"type":"record","name":"ColumnDef","fields":[{"name":"name","type":"bytes"},{"name":"validation_class","type":"string"},{"name":"index_type","type":[{"type":"enum","name":"IndexType","symbols":["KEYS","CUSTOM"],"aliases":["org.apache.cassandra.config.avro.IndexType"]},"null"]},{"name":"index_name","type":["string","null"]},{"name":"index_options","type":["null",{"type":"map","values":"string"}],"default":null}],"aliases":["org.apache.cassandra.config.avro.ColumnDef"]}},"null"]},{"name":"row_cache_provider","type":["string","null"],"default":"org.apache.cassandra.cache.ConcurrentLinkedHashCacheProvider"},{"name":"key_alias","type":["null","bytes"],"default":null},{"name":"compaction_strategy","type":["null","string"],"default":null},{"name":"compaction_strategy_options","type":["null",{"type":"map","values":"string"}],"default":null},{"name":"compression_options","type":["null",{"type":"map","values":"string"}],"default":null},{"name":"bloom_filter_fp_chance","type":["double","null"]}],"aliases":["org.apache.cassandra.config.avro.CfDef"]}
        at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:212)
        at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
        at org.apache.avro.io.ResolvingDecoder.readFieldOrder(ResolvingDecoder.java:121)
        at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:138)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:114)
        at org.apache.avro.generic.GenericDatumReader.readArray(GenericDatumReader.java:192)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:116)
        at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:142)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:114)
        at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:105)
        at org.apache.cassandra.io.SerDeUtils.deserialize(SerDeUtils.java:60)
        at org.apache.cassandra.db.DefsTable.loadFromStorage(DefsTable.java:98)
        at org.apache.cassandra.config.DatabaseDescriptor.loadSchemas(DatabaseDescriptor.java:502)
        at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:179)
        at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:355)
        at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:107)

                
> BloomFilter FP ratio should be configurable or size-restricted some other way
> -----------------------------------------------------------------------------
>
>                 Key: CASSANDRA-3497
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3497
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Brandon Williams
>            Assignee: Yuki Morishita
>            Priority: Minor
>             Fix For: 1.0.7
>
>         Attachments: 3497-v3.txt, 3497-v4.txt, CASSANDRA-1.0-3497.txt
>
>
> When you have a live dc and purely analytical dc, in many situations you can have less
nodes on the analytical side, but end up getting restricted by having the BloomFilters in-memory,
even though you have absolutely no use for them.  It would be nice if you could reduce this
memory requirement by tuning the desired FP ratio, or even just disabling them altogether.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message