hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ahad Rana (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-6834) TFile.append compares initial key against null lastKey
Date Tue, 22 Jun 2010 07:24:56 GMT
TFile.append compares initial key against null lastKey  

                 Key: HADOOP-6834
                 URL: https://issues.apache.org/jira/browse/HADOOP-6834
             Project: Hadoop Common
          Issue Type: Bug
          Components: io
    Affects Versions: 0.20.2, 0.20.1
            Reporter: Ahad Rana

The following code in TFile.KeyReigster.close: 

            byte[] lastKey = lastKeyBufferOS.getBuffer();
            int lastLen = lastKeyBufferOS.size();
            if (tfileMeta.getComparator().compare(key, 0, len, lastKey, 0,
                lastLen) < 0) {
              throw new IOException("Keys are not added in sorted order");

compares the initial  key (passed in via  TFile.Writer.append) against a technically NULL
lastKey. lastKey is not initialized until after the first call to TFile.Writer.append. The
underlying RawComparator interface used for comparisons does not stipulate the proper behavior
when either length 1  or length 2 is zero. In the case of LongWritable, its WritableComparator
implementation does an unsafe read on the passed in byte arrays b1 and b2. Since TFile pre-allocates
the buffer used for storing lastKey, this passes a valid buffer with zero count to LongWritable's
comparator, which ignores length and thus produces incorrect results. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message