Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 30 Aug 2016 02:17:21 +0000 (UTC)
From: "Vladimir Rodionov (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13001168.1472522953000.442928.1472523441441@Atlassian.JIRA>
In-Reply-To: <JIRA.13001168.1472522953000@Atlassian.JIRA>
References: <JIRA.13001168.1472522953000@Atlassian.JIRA> <JIRA.13001168.1472522953277@arcas>
Subject: [jira] [Commented] (HBASE-16527) IOExceptions from DFS client still
 can cause CatalogJanitor to delete referenced files
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 30 Aug 2016 02:17:23 -0000


    [ https://issues.apache.org/jira/browse/HBASE-16527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15447743#comment-15447743 ] 

Vladimir Rodionov commented on HBASE-16527:
-------------------------------------------

CatalogJanitor:
{code}
  Pair<Boolean, Boolean> checkDaughterInFs(final HRegionInfo parent, final HRegionInfo daughter)

SKIP-SKIP
    boolean references = false;
    HTableDescriptor parentDescriptor = getTableDescriptor(parent.getTable());
    for (HColumnDescriptor family: parentDescriptor.getFamilies()) {
      if ((references = regionFs.hasReferences(family.getNameAsString()))) {
        break;
      }
    }
    return new Pair<Boolean, Boolean>(Boolean.TRUE, Boolean.valueOf(references));
{code}

HRegionFileSystem:
{code}
  /**
   * Returns true if the specified family has reference files
   * @param familyName Column Family Name
   * @return true if family contains reference files
   * @throws IOException
   */
  public boolean hasReferences(final String familyName) throws IOException {
    FileStatus[] files = FSUtils.listStatus(fs, getStoreDir(familyName),
        new FSUtils.ReferenceFileFilter(fs));
    return files != null && files.length > 0;
  }
{code}

 FSUtils.ReferenceFileFilter:

{code}
    @Override
    protected boolean accept(Path p, @CheckForNull Boolean isDir) {
      if (!StoreFileInfo.isReference(p)) {
        return false;
      }

      try {
        // only files can be references.
        return isFile(fs, isDir, p);
      } catch (IOException ioe) {
        // Maybe the file was moved or the fs was disconnected.
        LOG.warn("Skipping file " + p +" due to IOException", ioe);
        return false;
      }
    }
{code}

isFile can throw IOException and we return false (not reference) - which is bad.


> IOExceptions from DFS client still can cause CatalogJanitor to delete referenced files
> --------------------------------------------------------------------------------------
>
>                 Key: HBASE-16527
>                 URL: https://issues.apache.org/jira/browse/HBASE-16527
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>             Fix For: 2.0.0
>
>
> that was fixed partially in HBASE-13331, but issue still exists , now a little bit deeper in the code.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)