hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars George (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4061) getTableDirs is missing directories to skip
Date Tue, 05 Jul 2011 10:05:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13059803#comment-13059803
] 

Lars George commented on HBASE-4061:
------------------------------------

Here is a patch that fixes it, but has to hardcode (*gasp*) the "splitlog" name, courtesy
of ZKSplitLog.getSplitLogDir(). :) This should be in HConstants.java - but I would still prefer
to change this to ".splitlog" and make this a rule that all non-table dirs/files are dot prefixed.

{noformat}
--- src/main/java/org/apache/hadoop/hbase/util/FSUtils.java     (revision 1139502)
+++ src/main/java/org/apache/hadoop/hbase/util/FSUtils.java     (working copy)
@@ -712,7 +712,8 @@
     public boolean accept(Path p) {
       boolean isdir = false;
       try {
-        isdir = this.fs.getFileStatus(p).isDir();
+        isdir = this.fs.getFileStatus(p).isDir() &&
+          !p.getName().startsWith(".");
       } catch (IOException e) {
         e.printStackTrace();
       }
@@ -854,6 +855,7 @@
       if (tableName.equals(HConstants.HREGION_LOGDIR_NAME) ||
           tableName.equals(Bytes.toString(HConstants.ROOT_TABLE_NAME)) ||
           tableName.equals(Bytes.toString(HConstants.META_TABLE_NAME)) ||
+          tableName.equals("splitlog") ||
           tableName.equals(HConstants.HREGION_OLDLOGDIR_NAME) ) {
         continue;
       }
{noformat}



> getTableDirs is missing directories to skip
> -------------------------------------------
>
>                 Key: HBASE-4061
>                 URL: https://issues.apache.org/jira/browse/HBASE-4061
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>    Affects Versions: 0.92.0
>            Reporter: Lars George
>             Fix For: 0.92.0
>
>
> The getTableDirs() is missing extra checks:
> {code}
>   public static List<Path> getTableDirs(final FileSystem fs, final Path rootdir)
>   throws IOException {
>     // presumes any directory under hbase.rootdir is a table
>     FileStatus [] dirs = fs.listStatus(rootdir, new DirFilter(fs));
>     List<Path> tabledirs = new ArrayList<Path>(dirs.length);
>     for (FileStatus dir: dirs) {
>       Path p = dir.getPath();
>       String tableName = p.getName();
>       if (tableName.equals(HConstants.HREGION_LOGDIR_NAME) ||
>           tableName.equals(Bytes.toString(HConstants.ROOT_TABLE_NAME)) ||
>           tableName.equals(Bytes.toString(HConstants.META_TABLE_NAME)) ||
>           tableName.equals(HConstants.HREGION_OLDLOGDIR_NAME) ) {
>         continue;
>       }
>       tabledirs.add(p);
>     }
>     return tabledirs;
>   }
> {code}
> It needs to also skip 
> * .tmp
> * .corrupt
> * splitlog
> A broader check should be performed to make sure it is all covered.
> The missing .corrupt check causes for example:
> {noformat}
> 2011-07-05 11:34:33,364 WARN org.apache.hadoop.hbase.master.HMaster: Failed getting all
descriptors
> java.io.FileNotFoundException: No status for hdfs://localhost:8020/hbase/.corrupt
>         at org.apache.hadoop.hbase.util.FSUtils.getTableInfoModtime(FSUtils.java:888)
>         at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:122)
>         at org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:149)
>         at org.apache.hadoop.hbase.master.HMaster.getHTableDescriptors(HMaster.java:1429)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:312)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1065)
> {noformat}
> Not sure yet why others do not have this issue, could be me being on trunk and fiddling?

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message