accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-1946) Include dfs.datanode.synconclose in hdfs configuration documentation
Date Mon, 02 Dec 2013 19:02:35 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13836795#comment-13836795
] 

Josh Elser commented on ACCUMULO-1946:
--------------------------------------

Yes, I would assume that it does slow down writes to some extent. I thought briefly about
adding a note about this potentially being unnecessary if you're running with UPCs or redundant
power supplies, but decided against it.

Do [~kturner] or [~ecn] have an idea for the amount of penalty incurred? Looking at HBASE-5954,
Lars reported a little under 50% penalty (if I'm reading that correctly, and those numbers
didn't change in the end). Personally, I'd much rather warn that they should turn it on, and
let someone turn it off if they're comfortable. What do you think about making a note about
being able to turn it off when you have more robust hardware power failures? I don't necessarily
want to say "we recommend this, but it's going to make it slower". I doesn't seem like the
norm to accept data corruption for a relatively small improvement in performance, especially
when we think in terms of commodity hardware.

Thoughts?

> Include dfs.datanode.synconclose in hdfs configuration documentation
> --------------------------------------------------------------------
>
>                 Key: ACCUMULO-1946
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1946
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: docs
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: 1.5.1, 1.6.0
>
>
> We should be including some writeup about dfs.datanode.synconclose in our documentation
surrounding the HDFS configuration as it better ensures that data is lost in the face of hard
shutdown (power loss) of the datanode process.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message