cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jon Haddad (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12979) checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
Date Thu, 05 Jan 2017 23:09:58 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802816#comment-15802816
] 

Jon Haddad commented on CASSANDRA-12979:
----------------------------------------

Trunk functions correctly: 

{code}
DEBUG [MemtableFlushWriter:1] 2017-01-05 15:07:58,036 ColumnFamilyStore.java:1194 - Flushed
to [BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db')]
(1 sstables, 964.442KiB), biggest 964.442KiB, smallest 964.442KiB
DEBUG [CompactionExecutor:2] 2017-01-05 15:07:59,051 CompactionTask.java:255 - Compacted (c4056ba0-d39b-11e6-b610-ef0dd3682585)
32 sstables to [/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-339-big,]
to level=0.  27.586MiB to 28.058MiB (~101% of original) in 14,704ms.  Read Throughput = 1.876MiB/s,
Write Throughput = 1.908MiB/s, Row Throughput = ~8,580/s.  128,704 total partitions merged
to 128,704.  Partition merge counts were {1:128704, }
DEBUG [CompactionExecutor:1] 2017-01-05 15:08:01,743 CompactionTask.java:255 - Compacted (c64bb220-d39b-11e6-b610-ef0dd3682585)
29 sstables to [/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-360-big,]
to level=0.  63.751MiB to 64.450MiB (~101% of original) in 13,580ms.  Read Throughput = 4.694MiB/s,
Write Throughput = 4.746MiB/s, Row Throughput = ~20,991/s.  293,883 total partitions merged
to 293,883.  Partition merge counts were {1:293883, }
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,743 CompactionTask.java:94 - insufficient
space to compact all requested files. 27.529812MB required, BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-406-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-385-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-387-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-389-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-381-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-370-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-372-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-375-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-393-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-391-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-401-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-409-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-403-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-384-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-407-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-363-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-382-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-365-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-361-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-388-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-378-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-369-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-380-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-371-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-374-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-394-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-396-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-400-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-398-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-408-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-404-big-Data.db')
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,744 CompactionTask.java:354 - Not enough
space for compaction, 27.529812MB estimated.  Reducing scope.
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,744 CompactionTask.java:94 - insufficient
space to compact all requested files. 26.669067MB required, BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-385-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-387-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-389-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-381-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-370-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-372-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-375-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-393-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-391-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-401-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-409-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-403-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-384-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-407-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-363-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-382-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-365-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-361-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-388-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-378-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-369-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-380-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-371-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-374-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-394-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-396-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-400-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-398-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-408-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-404-big-Data.db')
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,745 CompactionTask.java:354 - Not enough
space for compaction, 26.669067MB estimated.  Reducing scope.
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,745 CompactionTask.java:94 - insufficient
space to compact all requested files. 25.808325MB required, BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-385-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-387-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-389-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-381-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-370-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-372-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-375-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-393-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-391-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-401-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-409-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-403-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-384-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-407-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-363-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-382-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-365-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-361-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-388-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-378-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-369-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-371-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-374-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-394-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-396-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-400-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-398-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-408-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-404-big-Data.db')
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,745 CompactionTask.java:354 - Not enough
space for compaction, 25.808325MB estimated.  Reducing scope.
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,746 CompactionTask.java:94 - insufficient
space to compact all requested files. 24.947624MB required, BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-385-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-387-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-389-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-381-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-370-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-372-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-375-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-393-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-391-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-401-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-409-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-403-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-384-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-407-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-363-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-382-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-365-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-361-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-388-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-378-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-369-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-371-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-374-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-394-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-396-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-400-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-408-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-404-big-Data.db')
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,746 CompactionTask.java:354 - Not enough
space for compaction, 24.947624MB estimated.  Reducing scope.
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,746 CompactionTask.java:94 - insufficient
space to compact all requested files. 24.086933MB required, BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-385-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-387-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-389-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-381-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-370-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-372-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-375-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-393-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-391-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-401-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-409-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-403-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-384-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-407-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-363-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-382-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-365-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-361-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-388-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-378-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-369-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-371-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-374-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-394-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-396-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-408-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-404-big-Data.db')
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,747 CompactionTask.java:354 - Not enough
space for compaction, 24.086933MB estimated.  Reducing scope.
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,747 CompactionTask.java:94 - insufficient
space to compact all requested files. 23.226252MB required, BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-385-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-387-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-389-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-381-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-370-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-372-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-375-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-393-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-391-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-401-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-409-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-403-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-384-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-407-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-363-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-382-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-365-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-361-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-388-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-378-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-369-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-371-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-374-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-394-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-408-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-404-big-Data.db')
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,747 CompactionTask.java:354 - Not enough
space for compaction, 23.226252MB estimated.  Reducing scope.
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,748 CompactionTask.java:94 - insufficient
space to compact all requested files. 22.36557MB required, BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-385-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-387-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-389-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-381-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-370-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-372-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-375-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-393-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-391-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-401-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-409-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-403-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-384-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-407-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-363-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-382-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-365-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-361-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-388-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-378-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-369-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-371-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-374-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-394-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-408-big-Data.db')
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,748 CompactionTask.java:354 - Not enough
space for compaction, 22.36557MB estimated.  Reducing scope.
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,748 CompactionTask.java:94 - insufficient
space to compact all requested files. 21.504894MB required, BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-385-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-387-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-389-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-381-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-370-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-372-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-375-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-393-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-401-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-411-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-409-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-403-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-384-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-407-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-363-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-382-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-365-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-361-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-388-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-378-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-369-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-371-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-374-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-394-big-Data.db'),
BigTableReader(path='/Volumes/tiny/keyspace1/standard1-771c8c10d39b11e6b610ef0dd3682585/mc-408-big-Data.db')
WARN  [CompactionExecutor:2] 2017-01-05 15:08:01,748 CompactionTask.java:354 - Not enough
space for compaction, 21.504894MB estimated.  Reducing scope.
{code}

> checkAvailableDiskSpace doesn't update expectedWriteSize when reducing thread scope
> -----------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-12979
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12979
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Jon Haddad
>            Assignee: Jon Haddad
>              Labels: compaction
>             Fix For: 2.2.9, 3.0.11, 4.0, 3.x
>
>         Attachments: 2.2-0001-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch,
30-Fixed-checkAvailableDiskSpace-to-properly-recalculat.patch
>
>
> If a compaction occurs that looks like it'll take up more space than remaining disk available,
the compaction manager attempts to reduce the scope of the compaction by calling {{reduceScopeForLimitedSpace()}}
repeatedly.  
> Unfortunately, the while loop passes the {{estimatedWriteSize}} calculated from the original
call to {{hasAvailableDiskSpace}}, so the comparisons that are done will always be against
the size of the original compaction, rather than the reduced scope one.
> Full method below:
> {code}
>     protected void checkAvailableDiskSpace(long estimatedSSTables, long expectedWriteSize)
>     {
>         if(!cfs.isCompactionDiskSpaceCheckEnabled() && compactionType == OperationType.COMPACTION)
>         {
>             logger.info("Compaction space check is disabled");
>             return;
>         }
>         while (!getDirectories().hasAvailableDiskSpace(estimatedSSTables, expectedWriteSize))
>         {
>             if (!reduceScopeForLimitedSpace())
>                 throw new RuntimeException(String.format("Not enough space for compaction,
estimated sstables = %d, expected write size = %d", estimatedSSTables, expectedWriteSize));
>       
>         }
>     }
> {code}
> I'm proposing to recalculate the {{estimatedSSTables}} and {{expectedWriteSize}} after
each iteration of {{reduceScopeForLimitedSpace}}.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message