hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2382) Don't rely on fs.getDefaultReplication() to roll HLogs
Date Mon, 29 Mar 2010 22:55:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12851147#action_12851147
] 

Jean-Daniel Cryans commented on HBASE-2382:
-------------------------------------------

I understand that well configured HBase clusters will normally be aware of the configured
HDFS replication factor. What I'm afraid of is that new users don't understand that they must
configure that on top of everything else and will complain about abysmal write speed since
the HLog will be rolled for every edit until they figure that part of the configuration. 

My best case scenario would be that we handle it for the user.

Else, an acceptable solution would be to ship HBase like Nicolas describes with dfs.replication
(already set to 1?) then we need to be very explicit in the documentation about how to set
this up for fully distributed operations. So, to close this jira, I think we need to add more
documentation in src/java/overview.html along with the new configuration file.

> Don't rely on fs.getDefaultReplication() to roll HLogs
> ------------------------------------------------------
>
>                 Key: HBASE-2382
>                 URL: https://issues.apache.org/jira/browse/HBASE-2382
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: Jean-Daniel Cryans
>            Assignee: Nicolas Spiegelberg
>             Fix For: 0.20.4, 0.21.0
>
>         Attachments: HBASE-2382-20.4.patch
>
>
> As I was commenting in HBASE-2234, using fs.getDefaultReplication() to roll HLogs if
they lose replicas isn't reliable since that value is client-side and unless HBase is configured
with it or has Hadoop's configurations on its classpath, it will do the wrong thing.
> Dhruba added:
> bq. Can we use <hlogpath>.getFiletatus().getReplication() instead of fs.getDefaltReplication()?
This will will ensure that we look at the repl factor of the precise file we are interested
in, rather than what the system-wide default value is.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message