hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15035) bulkloading hfiles with tags that require splits do not preserve tags
Date Mon, 28 Dec 2015 21:22:49 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15073161#comment-15073161
] 

Jonathan Hsieh commented on HBASE-15035:
----------------------------------------

Let me clarify -- yes there is a bug in v3 that pulled the didn't get includeTags.  However,
I did another version that didn't use the builder copy constructor (e.g. it just too the hfilercontext
from the reader and didn't go through the buggy code in the patch), traced it further down
to this line here where the underlaying reader is created [1].

I kept this patch focused for now but would be up for a discussion about whether or not it
makes sense to read from an hfile by default skipping over tags.

[1] https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java#L1805

> bulkloading hfiles with tags that require splits do not preserve tags
> ---------------------------------------------------------------------
>
>                 Key: HBASE-15035
>                 URL: https://issues.apache.org/jira/browse/HBASE-15035
>             Project: HBase
>          Issue Type: Bug
>          Components: HFile
>    Affects Versions: 0.98.0, 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0
>            Reporter: Jonathan Hsieh
>            Assignee: Jonathan Hsieh
>            Priority: Blocker
>             Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
>
>         Attachments: HBASE-15035-v2.patch, HBASE-15035-v3.patch, HBASE-15035-v4.patch,
HBASE-15035.patch
>
>
> When an hfile is created with cell tags present and it is bulk loaded into hbase the
tags will be present when loaded into a single region.  If the bulk load hfile spans multiple
regions, bulk load automatically splits the original hfile into a set of split hfiles corresponding
to each of the regions that the original covers.  
> Since 0.98, tags are not copied into the newly created split hfiles. (the default for
"includeTags" of the HFileContextBuilder [1] is uninitialized which defaults to false).  
This means acls, ttls, mob pointers and other tag stored values will not be bulk loaded in.
> [1]  https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileContextBuilder.java#L40



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message