hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: [jira] [Updated] (HBASE-15035) bulkloading hfiles with tags that require splits do not preserve tags
Date Mon, 28 Dec 2015 14:42:26 GMT
On my inspection, it was because the reader side created a hfile context
that relies on default settings.  V3 used the context from the half hfile
reader which in turn came from the hfile reader.  If you found something
else can you file a new jira to fix?

On Sunday, December 27, 2015, ramkrishna.s.vasudevan (JIRA) <jira@apache.org>
wrote:

>
>     [
> https://issues.apache.org/jira/browse/HBASE-15035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15072439#comment-15072439
> ]
>
> ramkrishna.s.vasudevan commented on HBASE-15035:
> ------------------------------------------------
>
> i just saw this commit and also found why the suggestion of checking the
> includeTags from reader's context did not work
> In the patch v3
> {code}
> this.includesTags = ctx.isCompressTags();
> {code}
> Instead of checking ctx.isIncludeTags it is iscompressTags and hence
> includeTags on the new Writer would have been false. The reason why I was
> thinking not to hard code it to true always is because if we make it true
> always then the writer will append some empty tag info like 0 tag length
> and that will be removed only on compaction.
>
> > bulkloading hfiles with tags that require splits do not preserve tags
> > ---------------------------------------------------------------------
> >
> >                 Key: HBASE-15035
> >                 URL: https://issues.apache.org/jira/browse/HBASE-15035
> >             Project: HBase
> >          Issue Type: Bug
> >          Components: HFile
> >    Affects Versions: 0.98.0, 1.0.0, 2.0.0, 1.1.0, 1.2.0, 1.3.0
> >            Reporter: Jonathan Hsieh
> >            Assignee: Jonathan Hsieh
> >            Priority: Blocker
> >             Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3, 0.98.17, 1.0.4
> >
> >         Attachments: HBASE-15035-v2.patch, HBASE-15035-v3.patch,
> HBASE-15035-v4.patch, HBASE-15035.patch
> >
> >
> > When an hfile is created with cell tags present and it is bulk loaded
> into hbase the tags will be present when loaded into a single region.  If
> the bulk load hfile spans multiple regions, bulk load automatically splits
> the original hfile into a set of split hfiles corresponding to each of the
> regions that the original covers.
> > Since 0.98, tags are not copied into the newly created split hfiles.
> (the default for "includeTags" of the HFileContextBuilder [1] is
> uninitialized which defaults to false).   This means acls, ttls, mob
> pointers and other tag stored values will not be bulk loaded in.
> > [1]
> https://github.com/apache/hbase/blob/master/hbase-common/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileContextBuilder.java#L40
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>


-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message