cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carl Yeksigian (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10347) Bulk Loader API could not tolerate even node failure
Date Wed, 30 Sep 2015 15:01:04 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14936949#comment-14936949
] 

Carl Yeksigian commented on CASSANDRA-10347:
--------------------------------------------

It makes sense to add the option, if you already know which hosts are going to fail. That
way you can specify both the hosts that don't need to be tried, and the max number of hosts
which can fail.

These options should be moved into {{ConfigHelper}} instead of {{AbstractBulkOutputFormat}},
with the rest of the Hadoop options. Otherwise, +1.

> Bulk Loader API could not tolerate even node failure
> ----------------------------------------------------
>
>                 Key: CASSANDRA-10347
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10347
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Shenghua Wan
>            Assignee: Paulo Motta
>             Fix For: 2.1.x, 2.2.x, 3.0.x
>
>         Attachments: AbstractBulkRecordWriter.java
>
>
> When user uses CqlBulkOutputFormat, it tries to stream to all the nodes in the token
range, which includes the dead nodes. Therefore, the stream failed. There was a design in
C* API to allow stream() method to have a list of ignore hosts, but it was not utilized.
> The empty-argument stream() method is called in all existing versions of C*, i.e.
> in v2.0.11, https://github.com/apache/cassandra/blob/cassandra-2.0.11/src/java/org/apache/cassandra/hadoop/AbstractBulkRecordWriter.java#L122
> in v2.1.5, https://github.com/apache/cassandra/blob/cassandra-2.1.5/src/java/org/apache/cassandra/hadoop/AbstractBulkRecordWriter.java#L122
> and current trunk branch https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/hadoop/cql3/CqlBulkRecordWriter.java#L241



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message