cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeremiah Jordan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-5898) nodetool cleanup interacting badly with LCS
Date Fri, 16 Aug 2013 20:33:48 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-5898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jeremiah Jordan updated CASSANDRA-5898:
---------------------------------------

    Description: 
Then run cassandra-stress read, cassandra-stress write and "nodetool cleanup" all at the same
time, a few times, it can cause overlapping row errors.

{noformat}
ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,174 LeveledManifest.java (line 223) At level
1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-245-Data.db')
[DecoratedKey(204853724659241194183955214890519, 30303132343830), DecoratedKey(69227335985728660912035125310473966323,
30393537373332)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-243-Data.db')
[DecoratedKey(217896711032704014921095870827202, 30333635363932), DecoratedKey(71430242198281555888954138354238066233,
30333035343132)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,174 LeveledManifest.java (line 223) At level
1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-213-Data.db')
[DecoratedKey(69231122848869715605691825585281602523, 30303236343834), DecoratedKey(138494249350583165433071928358975719316,
30393733303134)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-247-Data.db')
[DecoratedKey(71430378028108400200608251408083114019, 30383133313937), DecoratedKey(142788557025615975974639291059795386925,
30353437373934)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,175 LeveledManifest.java (line 223) At level
1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-249-Data.db')
[DecoratedKey(138495280291659088173025258556445559667, 30313039353935), DecoratedKey(170139918483433930784261689358244392849,
30313336303439)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-252-Data.db')
[DecoratedKey(142788613459989682219570222310727294191, 30383031333839), DecoratedKey(170140890916069420209456014750832266661,
30333731303535)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
ERROR [CompactionExecutor:3] 2013-08-16 19:22:28,912 LeveledManifest.java (line 223) At level
1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-271-Data.db')
[DecoratedKey(68545052253582928820015773344685671746, 30303739313938), DecoratedKey(119320297902762425433209003370359962009,
30373137323238)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-270-Data.db')
[DecoratedKey(69231122848869715605691825585281602523, 30303236343834), DecoratedKey(138494249350583165433071928358975719316,
30393733303134)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
{noformat}

Steps to repro:
run

# Run cassandra-stress -Z LeveledCompactionStrategy
# Run nodetool cleanup
# While cleanup is still running, run cassandra-stress -o write and cassandra-stress -o read
at the same time.

  was:
If you setup "Keyspace1.Standard1" as LCS with 50mb files:

{noformat}
create column family Standard1
  with column_type = 'Standard'
  and comparator = 'AsciiType'
  and default_validation_class = 'BytesType'
  and key_validation_class = 'BytesType'
  and read_repair_chance = 0.1
  and dclocal_read_repair_chance = 0.0
  and gc_grace = 864000
  and min_compaction_threshold = 4
  and max_compaction_threshold = 32
  and replicate_on_write = true
  and compaction_strategy = 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'
  and caching = 'KEYS_ONLY'
  and compaction_strategy_options = {'sstable_size_in_mb' : '50'};
{noformat}

Then run cassandra-stress read, cassandra-stress write and "nodetool cleanup" all at the same
time, a few times, it can cause overlapping row errors.

{noformat}
ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,174 LeveledManifest.java (line 223) At level
1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-245-Data.db')
[DecoratedKey(204853724659241194183955214890519, 30303132343830), DecoratedKey(69227335985728660912035125310473966323,
30393537373332)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-243-Data.db')
[DecoratedKey(217896711032704014921095870827202, 30333635363932), DecoratedKey(71430242198281555888954138354238066233,
30333035343132)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,174 LeveledManifest.java (line 223) At level
1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-213-Data.db')
[DecoratedKey(69231122848869715605691825585281602523, 30303236343834), DecoratedKey(138494249350583165433071928358975719316,
30393733303134)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-247-Data.db')
[DecoratedKey(71430378028108400200608251408083114019, 30383133313937), DecoratedKey(142788557025615975974639291059795386925,
30353437373934)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,175 LeveledManifest.java (line 223) At level
1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-249-Data.db')
[DecoratedKey(138495280291659088173025258556445559667, 30313039353935), DecoratedKey(170139918483433930784261689358244392849,
30313336303439)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-252-Data.db')
[DecoratedKey(142788613459989682219570222310727294191, 30383031333839), DecoratedKey(170140890916069420209456014750832266661,
30333731303535)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
ERROR [CompactionExecutor:3] 2013-08-16 19:22:28,912 LeveledManifest.java (line 223) At level
1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-271-Data.db')
[DecoratedKey(68545052253582928820015773344685671746, 30303739313938), DecoratedKey(119320297902762425433209003370359962009,
30373137323238)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-270-Data.db')
[DecoratedKey(69231122848869715605691825585281602523, 30303236343834), DecoratedKey(138494249350583165433071928358975719316,
30393733303134)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
{noformat}

    
> nodetool cleanup interacting badly with LCS
> -------------------------------------------
>
>                 Key: CASSANDRA-5898
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5898
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Jeremiah Jordan
>            Assignee: Yuki Morishita
>
> Then run cassandra-stress read, cassandra-stress write and "nodetool cleanup" all at
the same time, a few times, it can cause overlapping row errors.
> {noformat}
> ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,174 LeveledManifest.java (line 223)
At level 1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-245-Data.db')
[DecoratedKey(204853724659241194183955214890519, 30303132343830), DecoratedKey(69227335985728660912035125310473966323,
30393537373332)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-243-Data.db')
[DecoratedKey(217896711032704014921095870827202, 30333635363932), DecoratedKey(71430242198281555888954138354238066233,
30333035343132)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
> ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,174 LeveledManifest.java (line 223)
At level 1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-213-Data.db')
[DecoratedKey(69231122848869715605691825585281602523, 30303236343834), DecoratedKey(138494249350583165433071928358975719316,
30393733303134)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-247-Data.db')
[DecoratedKey(71430378028108400200608251408083114019, 30383133313937), DecoratedKey(142788557025615975974639291059795386925,
30353437373934)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
> ERROR [CompactionExecutor:3] 2013-08-16 19:20:26,175 LeveledManifest.java (line 223)
At level 1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-249-Data.db')
[DecoratedKey(138495280291659088173025258556445559667, 30313039353935), DecoratedKey(170139918483433930784261689358244392849,
30313336303439)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-252-Data.db')
[DecoratedKey(142788613459989682219570222310727294191, 30383031333839), DecoratedKey(170140890916069420209456014750832266661,
30333731303535)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
> ERROR [CompactionExecutor:3] 2013-08-16 19:22:28,912 LeveledManifest.java (line 223)
At level 1, SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-271-Data.db')
[DecoratedKey(68545052253582928820015773344685671746, 30303739313938), DecoratedKey(119320297902762425433209003370359962009,
30373137323238)] overlaps SSTableReader(path='/raid0/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-270-Data.db')
[DecoratedKey(69231122848869715605691825585281602523, 30303236343834), DecoratedKey(138494249350583165433071928358975719316,
30393733303134)].  This is caused by a bug in Cassandra 1.1.0 .. 1.1.3.  Sending back to L0.
 If you have not yet run scrub, you should do so since you may also have rows out-of-order
within an sstable
> {noformat}
> Steps to repro:
> run
> # Run cassandra-stress -Z LeveledCompactionStrategy
> # Run nodetool cleanup
> # While cleanup is still running, run cassandra-stress -o write and cassandra-stress
-o read at the same time.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message