cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brandon Williams (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-3203) Odd flush behavior
Date Wed, 14 Sep 2011 00:56:08 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104124#comment-13104124
] 

Brandon Williams commented on CASSANDRA-3203:
---------------------------------------------

Here is what I've noticed:

Things proceed normally for a while:
{noformat}
 INFO 00:50:16,801 Enqueuing flush of Memtable-Standard1@304913075(36443580/45554475 serialized/live
bytes, 714580 ops)
{noformat}

Eventually, something bad happens:
{noformat}
 INFO 00:51:00,001 flushing high-traffic column family ColumnFamilyStore(table='Keyspace1',
columnFamily='Counter1') (estimated 0 bytes)
 INFO 00:51:00,002 flushing high-traffic column family ColumnFamilyStore(table='Keyspace1',
columnFamily='Super1') (estimated 0 bytes)
 INFO 00:51:00,002 flushing high-traffic column family ColumnFamilyStore(table='Keyspace1',
columnFamily='SuperCounter1') (estimated 0 bytes)
 INFO 00:51:00,002 flushing high-traffic column family ColumnFamilyStore(table='Keyspace1',
columnFamily='Standard1') (estimated 88291581 bytes)
 INFO 00:51:00,004 Enqueuing flush of Memtable-Standard1@741697653(2646900/88291581 serialized/live
bytes, 51900 ops)
 INFO 00:51:00,004 flushing high-traffic column family ColumnFamilyStore(table='system', columnFamily='NodeIdInfo')
(estimated 0 bytes)
 INFO 00:51:00,004 flushing high-traffic column family ColumnFamilyStore(table='system', columnFamily='IndexInfo')
(estimated 0 bytes)
 INFO 00:51:00,004 flushing high-traffic column family ColumnFamilyStore(table='system', columnFamily='LocationInfo')
(estimated 87 bytes)
 INFO 00:51:00,005 Enqueuing flush of Memtable-LocationInfo@1706933590(70/87 serialized/live
bytes, 2 ops)
 INFO 00:51:00,005 flushing high-traffic column family ColumnFamilyStore(table='system', columnFamily='Migrations')
(estimated 0 bytes)
 INFO 00:51:00,006 flushing high-traffic column family ColumnFamilyStore(table='system', columnFamily='HintsColumnFamily')
(estimated 0 bytes)
 INFO 00:51:00,006 flushing high-traffic column family ColumnFamilyStore(table='system', columnFamily='Schema')
(estimated 0 bytes)
 INFO 00:51:00,006 estimated 0 bytes used by all memtables pre-flush
 INFO 00:51:00,008 flushing ColumnFamilyStore(table='Keyspace1', columnFamily='Standard1')
to free up 459320 bytes
 INFO 00:51:00,009 Enqueuing flush of Memtable-Standard1@456936648(14280/476332 serialized/live
bytes, 280 ops)
 INFO 00:51:00,010 flushing ColumnFamilyStore(table='system', columnFamily='Schema') to free
up 0 bytes
 INFO 00:51:00,010 flushing ColumnFamilyStore(table='system', columnFamily='HintsColumnFamily')
to free up 0 bytes
 INFO 00:51:00,011 flushing ColumnFamilyStore(table='system', columnFamily='Migrations') to
free up 0 bytes
 INFO 00:51:00,011 flushing ColumnFamilyStore(table='system', columnFamily='LocationInfo')
to free up 0 bytes
 INFO 00:51:00,012 flushing ColumnFamilyStore(table='system', columnFamily='IndexInfo') to
free up 0 bytes
 INFO 00:51:00,013 flushing ColumnFamilyStore(table='system', columnFamily='NodeIdInfo') to
free up 0 bytes
 INFO 00:51:00,013 flushing ColumnFamilyStore(table='Keyspace1', columnFamily='SuperCounter1')
to free up 0 bytes
 INFO 00:51:00,014 flushing ColumnFamilyStore(table='Keyspace1', columnFamily='Super1') to
free up 0 bytes
 INFO 00:51:00,014 flushing ColumnFamilyStore(table='Keyspace1', columnFamily='Counter1')
to free up 0 bytes
 INFO 00:51:01,016 flushing high-traffic column family ColumnFamilyStore(table='Keyspace1',
columnFamily='Counter1') (estimated 0 bytes)
 INFO 00:51:01,016 flushing high-traffic column family ColumnFamilyStore(table='Keyspace1',
columnFamily='Super1') (estimated 0 bytes)
 INFO 00:51:01,016 flushing high-traffic column family ColumnFamilyStore(table='Keyspace1',
columnFamily='SuperCounter1') (estimated 0 bytes)
 INFO 00:51:01,016 flushing high-traffic column family ColumnFamilyStore(table='Keyspace1',
columnFamily='Standard1') (estimated 83808954 bytes)
 INFO 00:51:01,017 Enqueuing flush of Memtable-Standard1@911886300(2512770/83817460 serialized/live
bytes, 49270 ops)
{noformat}

After this point, it stays bad:

{noformat}
 INFO 00:51:01,879 Enqueuing flush of Memtable-Standard1@937412989(2670870/96531021 serialized/live
bytes, 52370 ops)
 INFO 00:51:02,029 Enqueuing flush of Memtable-Standard1@282282499(2668065/88997573 serialized/live
bytes, 52315 ops)
{noformat}

> Odd flush behavior
> ------------------
>
>                 Key: CASSANDRA-3203
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3203
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: Brandon Williams
>            Priority: Critical
>             Fix For: 1.0.0
>
>
> Given the same workload against 0.8, trunk is creating more than twice the amount of
sstables.  Even though a uniform stress workload is being generated, flush size degrades quickly:
> {noformat}
>  INFO [NonPeriodicTasks:1] 2011-09-09 18:24:22,878 ColumnFamilyStore.java (line 658)
Enqueuing flush of Memtable-Standard1@2058235391(7741
> 035/110172631 serialized/live bytes, 151785 ops)
>  INFO [NonPeriodicTasks:1] 2011-09-09 18:24:24,888 ColumnFamilyStore.java (line 658)
Enqueuing flush of Memtable-Standard1@1520390052(3887
> 220/72403158 serialized/live bytes, 76220 ops)
>  INFO [NonPeriodicTasks:1] 2011-09-09 18:24:26,890 ColumnFamilyStore.java (line 658)
Enqueuing flush of Memtable-Standard1@1868496516(4097
> 085/76255481 serialized/live bytes, 80335 ops)
>  INFO [NonPeriodicTasks:1] 2011-09-09 18:24:28,893 ColumnFamilyStore.java (line 658)
Enqueuing flush of Memtable-Standard1@498232521(43513
> 20/80922269 serialized/live bytes, 85320 ops)
>  INFO [NonPeriodicTasks:1] 2011-09-09 18:24:29,895 ColumnFamilyStore.java (line 658)
Enqueuing flush of Memtable-Standard1@1592308290(2310
> 810/44514839 serialized/live bytes, 45310 ops)
>  INFO [NonPeriodicTasks:1] 2011-09-09 18:24:30,897 ColumnFamilyStore.java (line 658)
Enqueuing flush of Memtable-Standard1@775439677(22684
> 80/64984390 serialized/live bytes, 44480 ops)
>  INFO [NonPeriodicTasks:1] 2011-09-09 18:24:31,899 ColumnFamilyStore.java (line 658)
Enqueuing flush of Memtable-Standard1@928217914(26741
> 85/76231422 serialized/live bytes, 52435 ops)
>  INFO [NonPeriodicTasks:1] 2011-09-09 18:24:32,901 ColumnFamilyStore.java (line 658)
Enqueuing flush of Memtable-Standard1@158103119(27511
> 95/77317732 serialized/live bytes, 53945 ops)
>  INFO [NonPeriodicTasks:1] 2011-09-09 18:24:33,903 ColumnFamilyStore.java (line 658)
Enqueuing flush of Memtable-Standard1@2035169258(3132
> 420/88934701 serialized/live bytes, 61420 ops)
>  INFO [NonPeriodicTasks:1] 2011-09-09 18:24:34,905 ColumnFamilyStore.java (line 658)
Enqueuing flush of Memtable-Standard1@1097314626(2979
> 675/83651699 serialized/live bytes, 58425 ops)
> {noformat}
> The serialized to live size ratio appears completely out of whack.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message