cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Morton (Commented) (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-3870) Internal error processing batch_mutate: java.util.ConcurrentModificationException on CounterColumn
Date Tue, 21 Feb 2012 19:41:46 GMT


Aaron Morton commented on CASSANDRA-3870:

bq. I don't think that is the goal of that code. We already have code for that (make sure
a node don't get overwhelm writing hints locally) in sendToHintedEndpoints.

Missed the totalHintsInProgress check.

bq. So I'm wondering, do we really have a strong reason for waiting for hints during writes
in the first place.

IMHO no, other than CL ANY. I know it's different in 1.0 but HH provides weak guarantees.
Recording a hint locally does not guarantee that it will be applied to the endpoint before
the next read for the row. 

bq. I'm not saying the attached patch won't work, but it does help making the write path more
complicated and 'messy' that I'd like it to be


If we stick with waiting for hints I could change it to *not* wait for acknowledgement that
all hint futures collected by default. Counter mutations on the coordinator are the exception,
they can tell the handler to wait.

bq. We should raise a timeout exception if waiting for the hints to be finalized timeouts
inserting of throwing an assertion error. It is possible (at least in theory) for that to


bq. Not sure I understand dontBlockOnHints(). It seems to only skip the fact that we signal
hintsFinalized. I think we should just call finalizeHints() when we don't want to block on

finalizeHints() stops modifications to the hints collection, so calling addFutureForHint()
after it will raise an exception. dontBlockOnHints() tells get() to ignore the hints collection
and prevents finalizeHints() from stopping modification to the hints collection. 

This is to support sendToHintedEndpoints passing hint futures to the handler when using CL
ONE. I wanted calls to addFutureForHint() to always work unless finalizeHints() had been called.

> Internal error processing batch_mutate: java.util.ConcurrentModificationException on
> --------------------------------------------------------------------------------------------------
>                 Key: CASSANDRA-3870
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 1.0.7
>         Environment: Debian 6.0 x64
> x64 Sun Java 6u26
> Cassandra 1.0.7
> 2 DCs, 1 ring/DC, 8 nodes/ring, RF=3/DC, Random partitioner.
> Disk access auto (mmap)
>            Reporter: Viktor Jevdokimov
>            Assignee: Aaron Morton
>              Labels: counters
>             Fix For: 1.0.8
>         Attachments: 3870.txt, cassandra-1.0-3870-V2.txt, cassandra-1.0-3870.txt
> Cassandra throws an exception below while performing batch_mutate with counter column
insertion mutation to increment column with 1:
> ERROR [Thrift:134] 2012-02-03 15:51:02,800 (line 3462) Internal error
processing batch_mutate
> java.util.ConcurrentModificationException
>         at java.util.AbstractList$Itr.checkForComodification(
>         at java.util.AbstractList$
>         at org.apache.cassandra.utils.FBUtilities.waitOnFutures(
>         at org.apache.cassandra.service.AbstractWriteResponseHandler.waitForHints(
>         at org.apache.cassandra.service.AbstractWriteResponseHandler.get(
>         at org.apache.cassandra.service.StorageProxy.mutate(
>         at org.apache.cassandra.thrift.CassandraServer.doInsert(
>         at org.apache.cassandra.thrift.CassandraServer.internal_batch_mutate(
>         at org.apache.cassandra.thrift.CassandraServer.batch_mutate(
>         at org.apache.cassandra.thrift.Cassandra$Processor$batch_mutate.process(
>         at org.apache.cassandra.thrift.Cassandra$Processor.process(
>         at org.apache.cassandra.thrift.CustomTThreadPoolServer$
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(
>         at java.util.concurrent.ThreadPoolExecutor$
>         at
> Column family definition:
> create column family CountersColumnFamily1
>   with column_type = 'Standard'
>   and comparator = 'BytesType'
>   and default_validation_class = 'BytesType'
>   and key_validation_class = 'BytesType'
>   and rows_cached = 1000000.0
>   and row_cache_save_period = 0
>   and row_cache_keys_to_save = 2147483647
>   and keys_cached = 0.0
>   and key_cache_save_period = 14400
>   and read_repair_chance = 0.1
>   and gc_grace = 43200
>   and min_compaction_threshold = 4
>   and max_compaction_threshold = 32
>   and replicate_on_write = true
>   and row_cache_provider = 'SerializingCacheProvider'
>   and compaction_strategy = 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy';

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message