cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9194) Delete-only workloads crash Cassandra
Date Tue, 21 Apr 2015 14:32:59 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505033#comment-14505033
] 

Benedict commented on CASSANDRA-9194:
-------------------------------------

[~mambocab]: thanks for being diligent about this. In 2.1 we _do_ account for its on-heap
size (if you check), which is what this bug is indicating we do not track. The deletion itself
is not counted towards the live data size though (i.e. the proportion of heap that is actually
data). This is because the space used is already used. I'm not sure we care enough to special
case this, as we need to check if the deletiontime _was_ live and is now _not_ live and add
8 in this case, but even this is kind of arbitrary. This live size in general is kind of an
arbitrary measure, so perhaps we can just add 8 to live bytes when we first insert a row,
since there will always be some overhead. 

Since the value we're reporting is kind of a fabrication, I'm sanguine about leaving it as
is, or changing it to some other arbitrary value. [~jbellis] got any opinion either way, since
you +1'd the first patch?

> Delete-only workloads crash Cassandra
> -------------------------------------
>
>                 Key: CASSANDRA-9194
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9194
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: 2.0.14
>            Reporter: Robert Wille
>            Assignee: Benedict
>             Fix For: 2.0.15
>
>         Attachments: 9194.txt
>
>
> The size of a tombstone is not properly accounted for in the memtable. A memtable which
has only tombstones will never get flushed. It will grow until the JVM runs out of memory.
The following program easily demonstrates the problem.
> {code}
> 		Cluster.Builder builder = Cluster.builder();
> 		
> 		Cluster c = builder.addContactPoints("cas121.devf3.com").build();
> 		
> 		Session s = c.connect();
> 			
> 		s.execute("CREATE KEYSPACE IF NOT EXISTS test WITH replication = { 'class' : 'SimpleStrategy',
'replication_factor' : 1 }");
> 		s.execute("CREATE TABLE IF NOT EXISTS test.test(id INT PRIMARY KEY)");
> 		PreparedStatement stmt = s.prepare("DELETE FROM test.test WHERE id = :id");
> 		int id = 0;
> 		
> 		while (true)
> 		{
> 			s.execute(stmt.bind(id));
> 			
> 			id++;
> 		}{code}
> This program should run forever, but eventually Cassandra runs out of heap and craps
out. You needn't wait for Cassandra to crash. If you run "nodetool cfstats test.test" while
it is running, you'll see Memtable cell count grow, but Memtable data size will remain 0.
> This issue was fixed once before. I received a patch for version 2.0.5 (I believe), which
contained the fix, but the fix has apparently been lost, because it is clearly broken, and
I don't see the fix in the change logs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message