hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apekshit Sharma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7105) RS throws NPE on forcing compaction from HBase shell on a single bulk imported file.
Date Sat, 18 Jul 2015 03:24:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632239#comment-14632239

Apekshit Sharma commented on HBASE-7105:

[~enis] [~clehene]
I also don't think that this is specifically related to bulk files in any way. Looks like
the main issue here is that 'majorCompacted' field's value requires the reader to be open
and is invalid until then, however the functions to get the value and open the reader does
not reflect that dependence in relationship and can be called independently. This has led
to code like 
  public boolean isMajorCompaction() throws IOException {
    for (StoreFile sf : this.storeEngine.getStoreFileManager().getStorefiles()) {
      // TODO: what are these reader checks all over the place?
      if (sf.getReader() == null) {
        LOG.debug("StoreFile " + sf + " has null Reader");
        return false;
    return storeEngine.getCompactionPolicy().isMajorCompaction(

The fix suggested in description seems like a bad thing to do because if createReader() isn't
called first for whatever reason, it may incorrectly advertise a store file as non-major when
it in fact was a major compaction file. At least current code fails correctly.

* open the reader explicitly if this.reader == null when isMajorCompacted() is called. However,
this will make subsequent createReader() call to throw IllegalAccessError. (my choice would
be keeping existing code than doing this)
* move the function to Reader class. (i feel like this will be best thing to do)

> RS throws NPE on forcing compaction from HBase shell on a single bulk imported file.
> ------------------------------------------------------------------------------------
>                 Key: HBASE-7105
>                 URL: https://issues.apache.org/jira/browse/HBASE-7105
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver
>            Reporter: Karthik Ranganathan
>            Assignee: Cosmin Lehene
>             Fix For: 2.0.0, 1.1.2, 1.3.0, 1.2.1, 1.0.3
>         Attachments: 0001-HBASE-7105-RS-throws-NPE-on-forcing-compaction-from-.patch,
> In StoreFile, we have:
> private AtomicBoolean majorCompaction = null;
> In StoreFile.open(), we do:
> b = metadataMap.get(MAJOR_COMPACTION_KEY);
> if (b != null) {
>   // init majorCompaction variable
> }
> Because the file was bulk imported, this is never initialized. Any subsequent call to
isMajorCompaction() NPE's.
> Fix is to initialize it to false.

This message was sent by Atlassian JIRA

View raw message