cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CASSANDRA-8409) Node generating a huge number of tiny sstable_activity flushes
Date Tue, 30 Dec 2014 14:51:13 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-8409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Benedict resolved CASSANDRA-8409.
---------------------------------
    Resolution: Duplicate

> Node generating a huge number of tiny sstable_activity flushes
> --------------------------------------------------------------
>
>                 Key: CASSANDRA-8409
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8409
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: Cassandra 2.1.0, Oracle JDK 1.8.0_25, Ubuntu 12.04
>            Reporter: Fred Wulff
>             Fix For: 2.1.3
>
>         Attachments: system-sstable_activity-ka-67802-Data.db
>
>
> On one of my nodes, I’m seeing hundreds per second of “INFO  21:28:05 Enqueuing flush
of sstable_activity: 0 (0%) on-heap, 33 (0%) off-heap”. tpstats shows a steadily climbing
# of pending MemtableFlushWriter/MemtablePostFlush until the node OOMs. When the flushes actually
happen the sstable written is invariably 121 bytes. I’m writing pretty aggressively to one
of my user tables (sev.mdb_group_pit), but that table's flushing behavior seems reasonable.
> tpstats:
> {quote}
> frew@hostname:~/s_dist/apache-cassandra-2.1.0$ bin/nodetool -h hostname tpstats
> Pool Name                    Active   Pending      Completed   Blocked  All time blocked
> MutationStage                   128      4429          36810         0              
  0
> ReadStage                         0         0           1205         0              
  0
> RequestResponseStage              0         0          24910         0              
  0
> ReadRepairStage                   0         0             26         0              
  0
> CounterMutationStage              0         0              0         0              
  0
> MiscStage                         0         0              0         0              
  0
> HintedHandoff                     2         2              9         0              
  0
> GossipStage                       0         0           5157         0              
  0
> CacheCleanupExecutor              0         0              0         0              
  0
> InternalResponseStage             0         0              0         0              
  0
> CommitLogArchiver                 0         0              0         0              
  0
> CompactionExecutor                4        28            429         0              
  0
> ValidationExecutor                0         0              0         0              
  0
> MigrationStage                    0         0              0         0              
  0
> AntiEntropyStage                  0         0              0         0              
  0
> PendingRangeCalculator            0         0             11         0              
  0
> MemtableFlushWriter               8     38644           8987         0              
  0
> MemtablePostFlush                 1     38940           8735         0              
  0
> MemtableReclaimMemory             0         0           8987         0              
  0
> Message type           Dropped
> READ                         0
> RANGE_SLICE                  0
> _TRACE                       0
> MUTATION                 10457
> COUNTER_MUTATION             0
> BINARY                       0
> REQUEST_RESPONSE             0
> PAGED_RANGE                  0
> READ_REPAIR                208
> {quote}
> I've attached one of the produced sstables.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message