hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4061) getTableDirs is missing directories to skip
Date Wed, 06 Jul 2011 20:41:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13060821#comment-13060821
] 

jiraposter@reviews.apache.org commented on HBASE-4061:
------------------------------------------------------


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1018/#review977
-----------------------------------------------------------

Ship it!


- Andrew


On 2011-07-06 20:39:39, Ted Yu wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/1018/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-07-06 20:39:39)
bq.  
bq.  
bq.  Review request for hbase.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  getTableDirs() should check for directories such as "splitlog"
bq.  
bq.  The patch reuses HConstants.HBASE_NON_USER_TABLE_DIRS
bq.  
bq.  
bq.  This addresses bug HBASE-4061.
bq.      https://issues.apache.org/jira/browse/HBASE-4061
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    /src/main/java/org/apache/hadoop/hbase/HConstants.java 1143201 
bq.    /src/main/java/org/apache/hadoop/hbase/util/FSUtils.java 1143525 
bq.    /src/main/java/org/apache/hadoop/hbase/zookeeper/ZKSplitLog.java 1143201 
bq.    /src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java 1143201 
bq.  
bq.  Diff: https://reviews.apache.org/r/1018/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Ran test suite.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Ted
bq.  
bq.



> getTableDirs is missing directories to skip
> -------------------------------------------
>
>                 Key: HBASE-4061
>                 URL: https://issues.apache.org/jira/browse/HBASE-4061
>             Project: HBase
>          Issue Type: Bug
>          Components: util
>    Affects Versions: 0.92.0
>            Reporter: Lars George
>            Assignee: Ted Yu
>             Fix For: 0.92.0
>
>         Attachments: 4061.txt
>
>
> The getTableDirs() is missing extra checks:
> {code}
>   public static List<Path> getTableDirs(final FileSystem fs, final Path rootdir)
>   throws IOException {
>     // presumes any directory under hbase.rootdir is a table
>     FileStatus [] dirs = fs.listStatus(rootdir, new DirFilter(fs));
>     List<Path> tabledirs = new ArrayList<Path>(dirs.length);
>     for (FileStatus dir: dirs) {
>       Path p = dir.getPath();
>       String tableName = p.getName();
>       if (tableName.equals(HConstants.HREGION_LOGDIR_NAME) ||
>           tableName.equals(Bytes.toString(HConstants.ROOT_TABLE_NAME)) ||
>           tableName.equals(Bytes.toString(HConstants.META_TABLE_NAME)) ||
>           tableName.equals(HConstants.HREGION_OLDLOGDIR_NAME) ) {
>         continue;
>       }
>       tabledirs.add(p);
>     }
>     return tabledirs;
>   }
> {code}
> It needs to also skip 
> * .tmp
> * .corrupt
> * splitlog
> A broader check should be performed to make sure it is all covered.
> The missing .corrupt check causes for example:
> {noformat}
> 2011-07-05 11:34:33,364 WARN org.apache.hadoop.hbase.master.HMaster: Failed getting all
descriptors
> java.io.FileNotFoundException: No status for hdfs://localhost:8020/hbase/.corrupt
>         at org.apache.hadoop.hbase.util.FSUtils.getTableInfoModtime(FSUtils.java:888)
>         at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:122)
>         at org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:149)
>         at org.apache.hadoop.hbase.master.HMaster.getHTableDescriptors(HMaster.java:1429)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:312)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1065)
> {noformat}
> Not sure yet why others do not have this issue, could be me being on trunk and fiddling?

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message