hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhihong Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6529) With HFile v2, the region server will always perform an extra copy of source files
Date Thu, 09 Aug 2012 15:52:19 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13431921#comment-13431921
] 

Zhihong Ted Yu commented on HBASE-6529:
---------------------------------------

bq.  The srcFs is a String passed into the server
srcPathStr is a String passed into the method. srcFs is the FileSystem corresponding to srcPathStr.

bq. you could do the same getBackingFs if srcFs is a HFileSystem.
Exactly. I suggest separating the logic of calling getBackingFs() and performing equality
check into a new method in, e.g., FSUtils

bq. HBase-6358 is about different FSs
Fixes for both JIRAs center around the following line in Store.java:
{code}
-    if (!srcFs.equals(fs)) {
{code}
Before HFileSystem came into play, the above check works adequately.
As identified above, one filesystem is wrapped in an HFileSystem, defeating the original intent
of the check.

Please correct me if I am wrong.
                
> With HFile v2, the region server will always perform an extra copy of source files
> ----------------------------------------------------------------------------------
>
>                 Key: HBASE-6529
>                 URL: https://issues.apache.org/jira/browse/HBASE-6529
>             Project: HBase
>          Issue Type: Bug
>          Components: performance, regionserver
>    Affects Versions: 0.94.0, 0.96.0
>            Reporter: Jason Dai
>         Attachments: hbase-6529.diff
>
>
> With HFile v2 implementation in HBase 0.94 & 0.96, the region server will use HFileSystem
as its {color:blue}fs{color}. When it performs bulk load in Store.bulkLoadHFile(), it checks
if its {color:blue}fs{color} is the same as {color:blue}srcFs{color}, which however will be
DistributedFileSystem. Consequently, it will always perform an extra copy of source files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message