cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-4305) CF serialization failure when working with custom secondary indices.
Date Fri, 08 Jun 2012 08:04:23 GMT


Sylvain Lebresne commented on CASSANDRA-4305:

No, the main argument is that we are reluctant to make the guarantee that RM object and their
CF objects are safe to be reused. And the reason for that is that maybe tomorrow we will add
something to the commit log code that will require having more than just the serialized mutation,
or maybe, as Jonathan said, we will be able to optimize so that serialize the RM directly
to the file (to avoid a copy), or maybe add some other code to Table.apply() (something related
to triggers for instance) that will want to pass the RM object to a separate thread.

The point is, assuming that a constucted RM (it's constructed as soon as you call apply basically)
is immutable makes it more easy to reason with in a concurrent context. And the fact that
we don't really need to use RM object concurrently today don't mean we won't do it tomorrow,
so saying that it is OK to mutate them will likely make our life harder at some point. And
if we don't guarantee that it's ok to mutate a constructed RM, then what is the point of making
it ok today, since we may break it tomorrow?

Furthermore, I have doubt that cloning the CF you're reusing before passing them to RM in
your 2ndary index code will have a measurable impact on performance (though if you have numbers
to show that it does make a noticeable difference, then it's a different discussion).

> CF serialization failure when working with custom secondary indices.
> --------------------------------------------------------------------
>                 Key: CASSANDRA-4305
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.10
>            Reporter: Pavel Yaskevich
>              Labels: datastax_qa
>         Attachments: CASSANDRA-4305.patch
> Assertion (below) was triggered when client was adding new rows to Solr-backed secondary
indices (1000-row batch without any timeout).
> {noformat}
> ERROR [COMMIT-LOG-WRITER] 2012-05-30 16:39:02,896 (line
139) Fatal exception in thread Thread[COMMIT-LOG-WRITER,5,main]
> java.lang.AssertionError: Final buffer length 176 to accomodate data size of 123 (predicted
87) for RowMutation(keyspace='solrTest1338395932411', key='6b6579383039', modifications=[ColumnFamily(cf1
>         at org.apache.cassandra.utils.FBUtilities.serialize(
>         at org.apache.cassandra.db.RowMutation.getSerializedBuffer(
>         at org.apache.cassandra.db.commitlog.CommitLogSegment.write(
>         at org.apache.cassandra.db.commitlog.CommitLog$
>         at org.apache.cassandra.db.commitlog.PeriodicCommitLogExecutorService$1.runMayThrow(
>         at
>         at
> {noformat}
> After investigation it was clear that it was happening because we were holding instances
of RowMutation queued to the addition to CommitLog to the actual "write" moment which is redundant.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message