accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-1637) Update HDFS append/sync precondition check for Hadoop 1.2
Date Wed, 16 Oct 2013 02:55:42 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-1637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13796360#comment-13796360
] 

Josh Elser commented on ACCUMULO-1637:
--------------------------------------

Currently trying to wrangle an "accurate" view on the Apache Hadoops. I don't think I'll be
able to make an assessment on what I think we should do until I have a complete picture.

I'm hoping that DFSConfigKeys will be a uniform place I can look for the defaults (or it will
at least match what the code might do implicitly, which I found one instance of already).

> Update HDFS append/sync precondition check for Hadoop 1.2
> ---------------------------------------------------------
>
>                 Key: ACCUMULO-1637
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1637
>             Project: Accumulo
>          Issue Type: Bug
>          Components: tserver
>    Affects Versions: 1.5.0
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Critical
>             Fix For: 1.5.1, 1.6.0
>
>
> Apache Hadoop 1.2.0 ships with the durable sync enabled by default and the support append
option marked as obsolete. Because of this, the check inside of TabletServer, meant to ensure
that the HDFS WAL can function properly, incorrectly fails as it doesn't know that dfs.durable.sync
is on by default.
> This can be worked around by specifying the old durable sync property in hdfs-site.xml:
> {noformat}
> <property>
>   <name>dfs.durable.sync</name>
>   <value>true</value>
> </property>
> {noformat}
> I'm not sure how to best way to address the differences between the newer and older versions
of Hadoop and their differing append support.
> Thanks to Carlos Mundi for pointing this out on user@a.a.o
> Using this table to track the presence of these variables and their default from hdfs/o/a/h/h/DFSConfigKeys
> ||Version||dfs.support.append||dfs.durable.sync||
> |0.20.205.0|default:false|not present|
> |0.23.x|default:true|not present|
> |1.0.x|default:false|not present|
> |1.1.x|not present|default:true|
> |1.1.2|not present|not present|
> |2.0.x|default:true|not present|
> |2.1.x|default:true|not present|
> |2.2.0|default:true|not present|



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message