cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood (JIRA)" <j...@apache.org>
Subject [jira] Reopened: (CASSANDRA-1264) Please add the ability to provide retry logic for the ColumnFamilyRecordWriter and ColumnFamilyRecordReader
Date Fri, 03 Sep 2010 06:20:33 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-1264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Stu Hood reopened CASSANDRA-1264:
---------------------------------


I'm reopening this because at the very least we would want to try all endpoints before giving
up. Also, the improvement on CASSANDRA-959 will minimize timeouts, but it doesn't help at
all with dead nodes, which we should discover and avoid.

> Please add the ability to provide retry logic for the ColumnFamilyRecordWriter and ColumnFamilyRecordReader
> -----------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-1264
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1264
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Hadoop
>            Reporter: James Mello
>
> Looking at the code for the ColumnFamilyRecordWriter and ColumnFamilyRecordReader, if
the service throws a backpressure exception, the map/reduce framework will end up having to
do retries to continue on the job. This failure scenario is pretty expensive and doesn't address
transient load exceptions. If the hadoop integration can add retry logic internally for these
scenarios, then the overall system would be greatly improved (IMO).
> A current idea on this would be to use a custom thread pool executor. Then the afterExecution
method could be overridden to do retry if a specific type of retry class is provided in the
job conf. I'll be trying to get a code example out soon as we need this internally, but if
it proves to be a reasonable model, we'd like to try to help out.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message