hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HBASE-8304) Bulkload fails to remove files if fs.default.name / fs.defaultFS is configured without default port
Date Mon, 10 Mar 2014 08:18:45 GMT

     [ https://issues.apache.org/jira/browse/HBASE-8304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Purtell resolved HBASE-8304.

       Resolution: Fixed
    Fix Version/s:     (was: 0.94.18)
                       (was: 0.96.2)

Re-resolving. These changes have already been committed for more than one day to trunk and
0.98. When I resolved this the last comment was 0.96 or 0.94 patches might come "later" at
some unspecified time. So now it's done. Leave this issue resolved and reopen exactly when
needed at commit time to 0.96 and 0.94, or file backport JIRAs. Leaving issues open after
commit is bad practice for two reasons: 1. JIRA searches return incorrect results, 2. JIRA
state becomes out of sync with repository (commit) state. Thanks in advance for your cooperation.

> Bulkload fails to remove files if fs.default.name / fs.defaultFS is configured without
default port
> ---------------------------------------------------------------------------------------------------
>                 Key: HBASE-8304
>                 URL: https://issues.apache.org/jira/browse/HBASE-8304
>             Project: HBase
>          Issue Type: Bug
>          Components: HFile, regionserver
>    Affects Versions: 0.94.5
>            Reporter: Raymond Liu
>            Assignee: haosdent
>              Labels: bulkloader
>             Fix For: 0.98.1, 0.99.0
>         Attachments: 0.94-8304.patch, 0.96-8304.patch, 8304-v4.patch, HBASE-8304-v2.patch,
HBASE-8304-v3.patch, HBASE-8304.patch
> When fs.default.name or fs.defaultFS in hadoop core-site.xml is configured as hdfs://ip,
and hbase.rootdir is configured as hdfs://ip:port/hbaserootdir where port is the hdfs namenode's
default port. the bulkload operation will not remove the file in bulk output dir. Store::bulkLoadHfile
will think hdfs:://ip and hdfs:://ip:port as different filesystem and go with copy approaching
instead of rename.
> The root cause is that hbase master will rewrite fs.default.name/fs.defaultFS according
to hbase.rootdir when regionserver started, thus, dest fs uri from the hregion will not matching
src fs uri passed from client.
> any suggestion what is the best approaching to fix this issue? 
> I kind of think that we could check for default port if src uri come without port info.

This message was sent by Atlassian JIRA

View raw message