hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8435) createNonRecursive support needed in WebHdfsFileSystem to support HBase
Date Thu, 13 Aug 2015 22:17:46 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jakob Homan updated HDFS-8435:
    Attachment: HDFS-8435.003.patch

New patch that applies to both trunk and branch 2.  

The failed tests were because the default of createParent param in WebHDFS was being set to
false, but then not being used by the actual call and overridden to true in the create call
on the dfsclient.  I've fixed this to pay attention to the parameter and updated the spec
to be correct.

Good catch on the throw.  Removed.

I had played around with that uber test a bit.  Using the annotation loses the explicit method
about what went wrong on each test.  I put as much into the helper method as looked reasonable
(judgment call here); when I put more of the per-test logic into the helper (expected exception,
subsequent message), it got really crowded and ugly.  

> createNonRecursive support needed in WebHdfsFileSystem to support HBase
> -----------------------------------------------------------------------
>                 Key: HDFS-8435
>                 URL: https://issues.apache.org/jira/browse/HDFS-8435
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: webhdfs
>    Affects Versions: 2.6.0
>            Reporter: Vinoth Sathappan
>            Assignee: Jakob Homan
>         Attachments: HDFS-8435-branch-2.7.001.patch, HDFS-8435.001.patch, HDFS-8435.002.patch,
> The WebHdfsFileSystem implementation doesn't support createNonRecursive. HBase extensively
depends on that for proper functioning. Currently, when the region servers are started over
web hdfs, they crash due with -
> createNonRecursive unsupported for this filesystem class org.apache.hadoop.hdfs.web.SWebHdfsFileSystem
>                 at org.apache.hadoop.fs.FileSystem.createNonRecursive(FileSystem.java:1137)
>                 at org.apache.hadoop.fs.FileSystem.createNonRecursive(FileSystem.java:1112)
>                 at org.apache.hadoop.fs.FileSystem.createNonRecursive(FileSystem.java:1088)
>                 at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter.init(ProtobufLogWriter.java:85)
>                 at org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createWriter(HLogFactory.java:198)

This message was sent by Atlassian JIRA

View raw message