hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yong Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HDFS-3767) Finer grained locking in DN
Date Wed, 22 Jul 2015 06:53:06 GMT

     [ https://issues.apache.org/jira/browse/HDFS-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Yong Zhang reassigned HDFS-3767:

    Assignee: Yong Zhang

> Finer grained locking in DN
> ---------------------------
>                 Key: HDFS-3767
>                 URL: https://issues.apache.org/jira/browse/HDFS-3767
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: performance
>    Affects Versions: 3.0.0
>            Reporter: Todd Lipcon
>            Assignee: Yong Zhang
> In testing a high-write-throughput workload, I see the DN maintain good performance most
of the time, except that occasionally one thread will block for a few seconds in {{finalizeReplica}}.
It does so holding the FSDatasetImpl lock, which causes all other writer threads to block
behind it. HDFS-1148 (making it a rw lock) would help here, but a bigger help would be to
go do finer-grained locking (eg per block or per-subdir).

This message was sent by Atlassian JIRA

View raw message