hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-8464) FastDiffEncoder - valueOffset calculation is incorrect
Date Tue, 30 Apr 2013 07:06:17 GMT

     [ https://issues.apache.org/jira/browse/HBASE-8464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

ramkrishna.s.vasudevan updated HBASE-8464:
------------------------------------------

    Summary: FastDiffEncoder - valueOffset calculation is incorrect  (was: FastDiffEncoder
- valueLength calculation is incorrect)
    
> FastDiffEncoder - valueOffset calculation is incorrect
> ------------------------------------------------------
>
>                 Key: HBASE-8464
>                 URL: https://issues.apache.org/jira/browse/HBASE-8464
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.94.7, 0.95.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 0.98.0, 0.94.8, 0.95.1
>
>
> In FastDiffEncoder
> Inside compressSingleKeyValue()
> {code}
>  currentState.prevOffset = in.position();
>     int keyLength = in.getInt();
>     int valueOffset =
>         currentState.prevOffset + keyLength + KeyValue.ROW_OFFSET;
>     int valueLength = in.getInt();
>     byte flag = 0;
> {code}
> Before seeing the bug, whenever we write something into encoders, we take the ByteBuffer
that is created by Writer.append().
> This basically writes
> keyLength, valueLength, keyarray, valuearray, <memstoreTS> 
> Now consider a case where the keyarray size is 20 and valuearray size is 20.
> As per the above code for the first KV
> Read keyLength (4  bytes), value length (4 bytes).
> First time the prevOffset is 0 so our value Offset is - 0+20+8 =28.
> This is correct.
> After the first KV is read when we take up the next KV,
> Now the currentState.prevOffset => 28+20 = 48 (the value is also read)
> The above calculation will give us
> 28+20+8 = 56.
> But the bytebuffer has only 48 bytes in it.
> Why our testcases did not catch this bug?
> ========================================
> It is because in the TestDataBlockEncoders we create a ByteBuffer directly from the KVs
and we do not create the way the HFileWriterV2 does it.
> See RedundantKVGenerator.convertKvToByteBuffer().
> Pls correct me if am wrong.  I can provide a patch for the same if my above analysis
is correct.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message