hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ecl...@apache.org
Subject hbase git commit: HBASE-13331 Exceptions from DFS client can cause CatalogJanitor to delete referenced files
Date Wed, 25 Mar 2015 02:56:55 GMT
Repository: hbase
Updated Branches:
  refs/heads/branch-1.0 53c6a57d0 -> 885a170ec


HBASE-13331 Exceptions from DFS client can cause CatalogJanitor to delete referenced files

Summary:
CatalogJanitor#checkDaughterInFs assumes that there are no references
whenever HRegionFileSystem.openRegionFromFileSystem throws IOException.
Well Hadoop and HBase throw IOExceptions whenever someone looks in their general direction.

This patch explicitly checks if the directory exists. If it doesn't then it allows references
to be
deleted. All other exceptions cause CatalogJanitor to assume there are references

Test Plan: Unit tests.

Differential Revision: https://reviews.facebook.net/D35829


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/885a170e
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/885a170e
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/885a170e

Branch: refs/heads/branch-1.0
Commit: 885a170ec5c7307f5c76ce0aa99dee9360903839
Parents: 53c6a57
Author: Elliott Clark <eclark@apache.org>
Authored: Tue Mar 24 13:42:29 2015 -0700
Committer: Elliott Clark <eclark@apache.org>
Committed: Tue Mar 24 19:52:24 2015 -0700

----------------------------------------------------------------------
 .../hadoop/hbase/master/CatalogJanitor.java      | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hbase/blob/885a170e/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
----------------------------------------------------------------------
diff --git a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
index 9f71b90..e07f930 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
@@ -370,14 +370,27 @@ public class CatalogJanitor extends Chore {
     Path rootdir = this.services.getMasterFileSystem().getRootDir();
     Path tabledir = FSUtils.getTableDir(rootdir, daughter.getTable());
 
+    Path daughterRegionDir = new Path(tabledir, daughter.getEncodedName());
+
     HRegionFileSystem regionFs = null;
+
+    try {
+      if (!FSUtils.isExists(fs, daughterRegionDir)) {
+        return new Pair<Boolean, Boolean>(Boolean.FALSE, Boolean.FALSE);
+      }
+    } catch (IOException ioe) {
+      LOG.warn("Error trying to determine if daughter region exists, " +
+               "assuming exists and has references", ioe);
+      return new Pair<Boolean, Boolean>(Boolean.TRUE, Boolean.TRUE);
+    }
+
     try {
       regionFs = HRegionFileSystem.openRegionFromFileSystem(
           this.services.getConfiguration(), fs, tabledir, daughter, true);
     } catch (IOException e) {
-      LOG.warn("Daughter region does not exist: " + daughter.getEncodedName()
-        + ", parent is: " + parent.getEncodedName());
-      return new Pair<Boolean, Boolean>(Boolean.FALSE, Boolean.FALSE);
+      LOG.warn("Error trying to determine referenced files from : " + daughter.getEncodedName()
+          + ", to: " + parent.getEncodedName() + " assuming has references", e);
+      return new Pair<Boolean, Boolean>(Boolean.TRUE, Boolean.TRUE);
     }
 
     boolean references = false;


Mime
View raw message