hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12137) DN dataset lock should be fair
Date Fri, 14 Jul 2017 21:22:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16088100#comment-16088100
] 

Hudson commented on HDFS-12137:
-------------------------------

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12010 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12010/])
HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. (kihwal: rev 8d86a93915ee00318289535d9c78e48b75c8359d)
* (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java


> DN dataset lock should be fair
> ------------------------------
>
>                 Key: HDFS-12137
>                 URL: https://issues.apache.org/jira/browse/HDFS-12137
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode
>    Affects Versions: 2.8.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Critical
>             Fix For: 2.9.0, 3.0.0-beta1, 2.8.3
>
>         Attachments: HDFS-12137.branch-2.patch, HDFS-12137.trunk.patch, HDFS-12137.trunk.patch
>
>
> The dataset lock is very highly contended.  The unfair nature can be especially harmful
to the heartbeat handling.  Under high loads, partially expose by HDFS-12136 introducing disk
i/o within the lock, the heartbeat handling thread may process commands so slowly due to the
contention that the node becomes stale or falsely declared dead.  The unfair lock is not helping
and appears to be causing frequent starvation under load.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message