hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hong Tang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6834) TFile.append compares initial key against null lastKey
Date Tue, 22 Jun 2010 22:17:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881404#action_12881404
] 

Hong Tang commented on HADOOP-6834:
-----------------------------------

You should always submit the patch for trunk. If you are using older releases, you may also
backport the patch (attach here) and request it to be committed to older branches too.

> TFile.append compares initial key against null lastKey  
> --------------------------------------------------------
>
>                 Key: HADOOP-6834
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6834
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 0.20.1, 0.20.2
>            Reporter: Ahad Rana
>
> The following code in TFile.KeyReigster.close: 
>             byte[] lastKey = lastKeyBufferOS.getBuffer();
>             int lastLen = lastKeyBufferOS.size();
>             if (tfileMeta.getComparator().compare(key, 0, len, lastKey, 0,
>                 lastLen) < 0) {
>               throw new IOException("Keys are not added in sorted order");
>             }
> compares the initial  key (passed in via  TFile.Writer.append) against a technically
NULL lastKey. lastKey is not initialized until after the first call to TFile.Writer.append.
The underlying RawComparator interface used for comparisons does not stipulate the proper
behavior when either length 1  or length 2 is zero. In the case of LongWritable, its WritableComparator
implementation does an unsafe read on the passed in byte arrays b1 and b2. Since TFile pre-allocates
the buffer used for storing lastKey, this passes a valid buffer with zero count to LongWritable's
comparator, which ignores length and thus produces incorrect results. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message