hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-9870) HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format
Date Sat, 02 Nov 2013 04:09:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811863#comment-13811863
] 

Anoop Sam John edited comment on HBASE-9870 at 11/2/13 4:09 AM:
----------------------------------------------------------------

Yes when we open a HFile, we will use the encoding stored in FileInfo.  Down the line when
we read blocks of data there will be the check that you mentioned in the issue right?  That
we dont want different encoding scheme for onDisk and inCache (onDisk != NONE)
In case when we are altering a CF with changing encoding from one alogo to another this situation
arises.  My feeling is that is not been handled.  So this situation requires data encoded
in a file using one algo should get encoded using another algo and get stored in cache.

Pls correct me if I am wrong Jimmy.


was (Author: anoop.hbase):
Yes when we open a HFile, we will use the encoding stored in FileInfo.  Down the line when
we read blocks of data there will be the check that you mentioned in the issue right?  That
we dont want different encoding scheme for onDisk and inCache (onDisk != NONE)
In case when we are altering a CF with changing encoding from one alogo to another this situation
arises.  My feeling is that is not been handled.  So this situation requires data encoded
in a file using one algo should get encoded using another algo and get stored in cache.

> HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format
> -------------------------------------------------------------
>
>                 Key: HBASE-9870
>                 URL: https://issues.apache.org/jira/browse/HBASE-9870
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jimmy Xiang
>
> In this method, we have
> {code}
>     if (block.getBlockType() == BlockType.ENCODED_DATA) {
>       if (block.getDataBlockEncodingId() == onDisk.getId()) {
>         // The block is already in the desired in-cache encoding.
>         return block;
>       }
> {code}
> This assumes onDisk encoding is the same as that of inCache.  This is not true when we
change the encoding of a CF.  This could be one of the reasons I got data loss with online
encoding change?
> If I make sure onDisk == inCache all the time, my ITBLL with online encoding change worked
once for me.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message