cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuki Morishita (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4435) hints compaction loop over same sstable
Date Mon, 16 Jul 2012 15:24:36 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415216#comment-13415216
] 

Yuki Morishita commented on CASSANDRA-4435:
-------------------------------------------

So far I have not able to reproduce this(If you still have one of those SSTables, it may help
me figure out easier).

But I noticed from the log and the HHOM code, HHOM tries user defined compaction to compact
all SSTables at once after forcing memtable to flush.
Flushing triggers background compaction, so sometimes, it competes with user defined one.
You may notice from the line in above log,

bq.  INFO 22:14:48,500 SSTables for user defined compaction are already being compacted.

shows this situation that SSTables are already marked for compaction when running user defined.

It may be not the right solution here, but since recursive compaction only occurs when doing
background compaction, I think it is better to add an option not to submit background compaction
after flushing and use it here to make sure you only run user defined compaction.
                
> hints compaction loop over same sstable
> ---------------------------------------
>
>                 Key: CASSANDRA-4435
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4435
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.2
>            Reporter: Brandon Williams
>            Assignee: Yuki Morishita
>
> Noticed the following while testing something else:
> {noformat}
> INFO 22:14:48,496 Completed flushing /var/lib/cassandra/data/system/hints/system-hints-ia-1-Data.db
(109645 bytes) for commitlog position ReplayPosition(segmentId=9372773011543415, position=30358488)
>  INFO 22:14:48,498 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-1-Data.db')]
>  INFO 22:14:48,500 SSTables for user defined compaction are already being compacted.
>  INFO 22:14:48,500 Finished hinted handoff of 16893 rows to endpoint /10.179.64.227
>  INFO 22:14:48,658 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-2-Data.db,].
 109,645 to 899 (~0% of original) bytes for 1 keys at 0.005392MB/s.  Time: 159ms.
>  INFO 22:14:48,660 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-2-Data.db')]
>  INFO 22:14:48,668 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-3-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.107169MB/s.  Time: 8ms.
>  INFO 22:14:48,669 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-3-Data.db')]
>  INFO 22:14:48,679 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-4-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.095261MB/s.  Time: 9ms.
>  INFO 22:14:48,680 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-4-Data.db')]
>  INFO 22:14:48,697 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-5-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.050433MB/s.  Time: 17ms.
>  INFO 22:14:48,698 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-5-Data.db')]
>  INFO 22:14:48,714 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-6-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.053585MB/s.  Time: 16ms.
>  INFO 22:14:48,715 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-6-Data.db')]
>  INFO 22:14:48,722 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-7-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.122479MB/s.  Time: 7ms.
>  INFO 22:14:48,723 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-7-Data.db')]
>  INFO 22:14:48,736 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-8-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.065950MB/s.  Time: 13ms.
>  INFO 22:14:48,737 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-8-Data.db')]
>  INFO 22:14:48,744 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-9-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.122479MB/s.  Time: 7ms.
>  INFO 22:14:48,745 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-9-Data.db')]
>  INFO 22:14:48,753 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-10-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.122479MB/s.  Time: 7ms.
>  INFO 22:14:48,754 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-10-Data.db')]
>  INFO 22:14:48,761 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-11-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.122479MB/s.  Time: 7ms.
>  INFO 22:14:48,762 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-11-Data.db')]
>  INFO 22:14:48,775 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-12-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.065950MB/s.  Time: 13ms.
>  INFO 22:14:48,776 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-12-Data.db')]
>  INFO 22:14:48,783 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-13-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.122479MB/s.  Time: 7ms.
>  INFO 22:14:48,784 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-13-Data.db')]
>  INFO 22:14:48,792 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-14-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.122479MB/s.  Time: 7ms.
>  INFO 22:14:48,793 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-14-Data.db')]
>  INFO 22:14:48,800 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-15-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.122479MB/s.  Time: 7ms.
>  INFO 22:14:48,802 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-15-Data.db')]
>  INFO 22:14:48,809 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-16-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.122479MB/s.  Time: 7ms.
>  INFO 22:14:48,810 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-16-Data.db')]
>  INFO 22:14:48,826 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-17-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.053585MB/s.  Time: 16ms.
>  INFO 22:14:48,827 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-17-Data.db')]
>  INFO 22:14:48,834 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-18-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.142892MB/s.  Time: 6ms.
>  INFO 22:14:48,835 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-18-Data.db')]
>  INFO 22:14:48,842 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-19-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.142892MB/s.  Time: 6ms.
>  INFO 22:14:48,842 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-19-Data.db')]
>  INFO 22:14:48,850 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-20-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.142892MB/s.  Time: 6ms.
>  INFO 22:14:48,850 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-20-Data.db')]
>  INFO 22:14:48,867 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-21-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.053585MB/s.  Time: 16ms.
>  INFO 22:14:48,868 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-21-Data.db')]
>  INFO 22:14:48,876 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-22-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.142892MB/s.  Time: 6ms.
>  INFO 22:14:48,877 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-22-Data.db')]
>  INFO 22:14:48,884 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-23-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.122479MB/s.  Time: 7ms.
>  INFO 22:14:48,885 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-23-Data.db')]
>  INFO 22:14:48,893 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-24-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.107169MB/s.  Time: 8ms.
>  INFO 22:14:48,895 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-24-Data.db')]
>  INFO 22:14:48,901 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-25-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.142892MB/s.  Time: 6ms.
>  INFO 22:14:48,902 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-25-Data.db')]
>  INFO 22:14:48,910 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-26-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.142892MB/s.  Time: 6ms.
>  INFO 22:14:48,910 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-26-Data.db')]
>  INFO 22:14:48,926 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-27-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.057157MB/s.  Time: 15ms.
>  INFO 22:14:48,930 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-27-Data.db')]
>  INFO 22:14:48,938 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-28-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.107169MB/s.  Time: 8ms.
>  INFO 22:14:48,943 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-28-Data.db')]
>  INFO 22:14:48,949 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-29-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.142892MB/s.  Time: 6ms.
>  INFO 22:14:48,950 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-29-Data.db')]
>  INFO 22:14:48,966 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-30-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.053585MB/s.  Time: 16ms.
>  INFO 22:14:48,967 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-30-Data.db')]
>  INFO 22:14:48,974 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-31-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.142892MB/s.  Time: 6ms.
>  INFO 22:14:48,974 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-31-Data.db')]
>  INFO 22:14:48,980 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-32-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.142892MB/s.  Time: 6ms.
>  INFO 22:14:48,981 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-32-Data.db')]
>  INFO 22:14:48,988 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-33-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.142892MB/s.  Time: 6ms.
>  INFO 22:14:48,989 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-33-Data.db')]
>  INFO 22:14:48,995 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-34-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.171471MB/s.  Time: 5ms.
>  INFO 22:14:48,995 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-34-Data.db')]
>  INFO 22:14:49,002 Compacted to [/var/lib/cassandra/data/system/hints/system-hints-ia-35-Data.db,].
 899 to 899 (~100% of original) bytes for 1 keys at 0.171471MB/s.  Time: 5ms.
>  INFO 22:14:49,002 Compacting [SSTableReader(path='/var/lib/cassandra/data/system/hints/system-hints-ia-35-Data.db')]
> {noformat}
> After this, the loop stopped.  It also did not occur on another member which delivered
hints, so it may not be easy to replicate.  I suspect something related to CASSANDRA-3442
caused this, though the odd thing is there shouldn't even be a tombstone left.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message