hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9870) HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format
Date Fri, 01 Nov 2013 06:13:17 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13811072#comment-13811072
] 

Anoop Sam John commented on HBASE-9870:
---------------------------------------

Yes this is to avoid a decode and then encode in the read path which is costly.  At any point
of time there can be only same type of encoding for OnDisk and InCache for a CF.  (If OnDisk
!= None)
But when we alter the encoding for a CF!!! Yes at time also both in cache and on disk are
same.. But the existing data!!! On disk that can be in another encoding format..  So this
is an issue.

Very good finding Jimmy..

> HFileDataBlockEncoderImpl#diskToCacheFormat uses wrong format
> -------------------------------------------------------------
>
>                 Key: HBASE-9870
>                 URL: https://issues.apache.org/jira/browse/HBASE-9870
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Jimmy Xiang
>
> In this method, we have
> {code}
>     if (block.getBlockType() == BlockType.ENCODED_DATA) {
>       if (block.getDataBlockEncodingId() == onDisk.getId()) {
>         // The block is already in the desired in-cache encoding.
>         return block;
>       }
> {code}
> This assumes onDisk encoding is the same as that of inCache.  This is not true when we
change the encoding of a CF.  This could be one of the reasons I got data loss with online
encoding change?
> If I make sure onDisk == inCache all the time, my ITBLL with online encoding change worked
once for me.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message