hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-4350) Make enabling of stale marking on read and write paths independent
Date Sat, 02 Feb 2013 22:44:12 GMT

     [ https://issues.apache.org/jira/browse/HDFS-4350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Suresh Srinivas updated HDFS-4350:
----------------------------------

          Resolution: Fixed
       Fix Version/s: 2.0.3-alpha
                      1.2.0
    Target Version/s: 1.2.0, 2.0.3-alpha  (was: 1.2.0, 3.0.0)
        Release Note: 
This patch makes an incompatible configuration change, as described below:
In releases 1.1.0 and other point releases 1.1.x, the configuration parameter "dfs.namenode.check.stale.datanode"
could be used to turn on checking for the stale nodes. This configuration is no longer supported
in release 1.2.0 onwards and is renamed as "dfs.namenode.avoid.read.stale.datanode". 

How feature works and configuring this feature:
As described in HDFS-3703 release notes, datanode stale period can be configured using parameter
"dfs.namenode.stale.datanode.interval" in seconds (default value is 30 seconds). NameNode
can be configured to use this staleness information for reads using configuration "dfs.namenode.avoid.read.stale.datanode".
When this parameter is set to true, namenode picks a stale datanode as the last target to
read from when returning block locations for reads. Using staleness information for writes
is as described in the releases notes of HDFS-3912.

        Hadoop Flags: Incompatible change,Reviewed  (was: Incompatible change)
              Status: Resolved  (was: Patch Available)

I committed the patch to trunk, branch-2 and branch-1.

Thank you Andrew!
                
> Make enabling of stale marking on read and write paths independent
> ------------------------------------------------------------------
>
>                 Key: HDFS-4350
>                 URL: https://issues.apache.org/jira/browse/HDFS-4350
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 3.0.0
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>             Fix For: 1.2.0, 2.0.3-alpha
>
>         Attachments: hdfs-4350-1.patch, hdfs-4350-2.patch, hdfs-4350-3.patch, hdfs-4350-4.patch,
hdfs-4350-5.patch, hdfs-4350-6.patch, hdfs-4350-7.patch, hdfs-4350-branch-1-1.patch, hdfs-4350-branch-1-2.patch,
hdfs-4350-branch-1-3.patch, hdfs-4350.txt
>
>
> Marking of datanodes as stale for the read and write path was introduced in HDFS-3703
and HDFS-3912 respectively. This is enabled using two new keys, {{DFS_NAMENODE_CHECK_STALE_DATANODE_KEY}}
and {{DFS_NAMENODE_AVOID_STALE_DATANODE_FOR_WRITE_KEY}}. However, there currently exists a
dependency, since you cannot enable write marking without also enabling read marking, since
the first key enables both checking of staleness and read marking.
> I propose renaming the first key to {{DFS_NAMENODE_AVOID_STALE_DATANODE_FOR_READ_KEY}},
and make checking enabled if either of the keys are set. This will allow read and write marking
to be enabled independently.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message