hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raymond Liu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-8304) Bulkload fail to remove files if fs.default.name / fs.defaultFS is configured without default port.
Date Tue, 09 Apr 2013 08:14:15 GMT
Raymond Liu created HBASE-8304:
----------------------------------

             Summary: Bulkload fail to remove files if fs.default.name / fs.defaultFS is configured
without default port.
                 Key: HBASE-8304
                 URL: https://issues.apache.org/jira/browse/HBASE-8304
             Project: HBase
          Issue Type: Bug
          Components: HFile, regionserver
    Affects Versions: 0.94.5
            Reporter: Raymond Liu


When fs.default.name or fs.defaultFS in hadoop core-site.xml is configured as hdfs://ip, and
hbase.rootdir is configured as hdfs://ip:port/hbaserootdir where port is the hdfs namenode's
default port. the bulkload operation will not remove the file in bulk output dir. Store::bulkLoadHfile
will think hdfs:://ip and hdfs:://ip:port as different filesystem and go with copy approaching
instead of rename.

The root cause is that hbase master will rewrite fs.default.name/fs.defaultFS according to
hbase.rootdir when regionserver started, thus, dest fs uri from the hregion will not matching
src fs uri passed from client.

any suggestion what is the best approaching to fix this issue? 

I kind of think that we could check for default port if src uri come without port info.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message