cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lanny Ripple (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5504) Eternal iteration when using newer hadoop version due to next() call and empty key value
Date Wed, 24 Apr 2013 19:49:16 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640845#comment-13640845
] 

Lanny Ripple commented on CASSANDRA-5504:
-----------------------------------------

Instead of reverting the changes to TBinaryProtocol you probably need to use ConfigHelper
to set the thrift_framed-transport_size_in_mb and thrift_max_message_length_in_mb to much
larger values (if ConfigHelper is exposed for you).  These values, prior to 1.10, were ignored
(and a later version fixed a bug with getting them from ConfigHelper as well).  Setting the
values to 2047 and 2048 respectively got us working again.

Oleksandr -- patch2 works for us.  Thanks!
                
> Eternal iteration when using newer hadoop version due to next() call and empty key value
> ----------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-5504
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5504
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Hadoop
>    Affects Versions: 1.2.3
>            Reporter: Oleksandr Petrov
>            Priority: Critical
>         Attachments: patch2.diff, patch.diff
>
>
> Currently, when using newer hadoop versions, due to the call to 
> next(ByteBuffer key, SortedMap<ByteBuffer, IColumn> value)
> within ColumnFamilyRecordReader, because `key.clear();` is called, key is emptied. That
causes the StaticRowIterator and WideRowIterator to glitch, namely, when Iterables.getLast(rows).key
is called, key is already empty. This will cause Hadoop to request the same range again and
again all the time.
> Please see the attached patch/diff, it simply adds lastRowKey (ByteBuffer) and saves
it for the next iteration along with all the rows, this allows query for the next range to
be fully correct.
> This patch is branched from 1.2.3 version.
> Tested against Cassandra 1.2.3, with Hadoop 1.0.3, 1.0.4 and 0.20.2

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message