hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6529) With HFile v2, the region server will always perform an extra copy of source files
Date Thu, 09 Aug 2012 15:34:19 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431905#comment-13431905

stack commented on HBASE-6529:

bq. What if srcFs is an HFileSystem and fs is DistributedFileSystem ?

How could that happen?   HFileSystem is the fs used by the running HRegionServer.  The srcFs
is a String passed into the server... You'd have to do something perverse having that be an
HFileSystem specification?

To guard against such a thing happening, I suppose you could do the same getBackingFs if srcFs
is a HFileSystem.

bq. The root cause is that we cannot rely on hadoop semantics for judging equality between
HFileSystem and DistributedFileSystem.

How so?

HBase-6358 is about different FSs.  This issue seems to be about issue where the filesystems
are the same only the one used by the regionserver is wrapped in an HFileSystem ... so the
equals fails.
> With HFile v2, the region server will always perform an extra copy of source files
> ----------------------------------------------------------------------------------
>                 Key: HBASE-6529
>                 URL: https://issues.apache.org/jira/browse/HBASE-6529
>             Project: HBase
>          Issue Type: Bug
>          Components: performance, regionserver
>    Affects Versions: 0.94.0, 0.96.0
>            Reporter: Jason Dai
>         Attachments: hbase-6529.diff
> With HFile v2 implementation in HBase 0.94 & 0.96, the region server will use HFileSystem
as its {color:blue}fs{color}. When it performs bulk load in Store.bulkLoadHFile(), it checks
if its {color:blue}fs{color} is the same as {color:blue}srcFs{color}, which however will be
DistributedFileSystem. Consequently, it will always perform an extra copy of source files.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message