Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 1 Nov 2013 18:15:20 +0000 (UTC)
From: "Matt Corgan (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12676964.1383256627796.8028.1383329720435@arcas>
In-Reply-To: <JIRA.12676964.1383256627796@arcas>
References: <JIRA.12676964.1383256627796@arcas>
Subject: [jira] [Commented] (HBASE-9870)
 HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-9870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811523#comment-13811523 ] 

Matt Corgan commented on HBASE-9870:
------------------------------------

I forget the intricacies (or never understood them).  I think the facebook guys added this ability to have different on-disk/in-cache encodings because they wanted to slowly test encoding without changing/corrupting the on-disk format, but now this safety feature has become dangerous.  In my opinion, we should just remove that feature and always load from on-disk to in-cache without changing the format.  To change the encoding, you would run a compaction.

> HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format
> -------------------------------------------------------------
>
>                 Key: HBASE-9870
>                 URL: https://issues.apache.org/jira/browse/HBASE-9870
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jimmy Xiang
>
> In this method, we have
> {code}
>     if (block.getBlockType() == BlockType.ENCODED_DATA) {
>       if (block.getDataBlockEncodingId() == onDisk.getId()) {
>         // The block is already in the desired in-cache encoding.
>         return block;
>       }
> {code}
> This assumes onDisk encoding is the same as that of inCache.  This is not true when we change the encoding of a CF.  This could be one of the reasons I got data loss with online encoding change?
> If I make sure onDisk == inCache all the time, my ITBLL with online encoding change worked once for me.


--
This message was sent by Atlassian JIRA
(v6.1#6144)