hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1148) Convert FSDataset to ReadWriteLock
Date Mon, 06 Jun 2011 05:03:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044712#comment-13044712

Todd Lipcon commented on HDFS-1148:

As I was updating HDFS-941 to trunk tonight, I took the opportunity to look into the blocking
behavior again. While running TestParallelRead (with N_ITERATIONS bumped up 10x) I ran:
$ while true ; do jstack 3378 | grep -A2 BLOCK >> /tmp/blocked ; done
and then when it was done:
$ grep 'at ' /tmp/blocked  | sort | uniq -c | sort -nk1
      1         at java.lang.Object.wait(Native Method)
      6         at org.apache.hadoop.hdfs.DFSInputStream.getBlockRange(DFSInputStream.java:313)
     27         at org.apache.hadoop.hdfs.TestParallelRead$ReadWorker.read(TestParallelRead.java:142)
    137         at org.apache.hadoop.hdfs.server.datanode.FSDataset.getBlockInputStream(FSDataset.java:1286)
    183         at org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:100)
    220         at org.apache.hadoop.hdfs.DFSInputStream.getFileLength(DFSInputStream.java:206)
    251         at org.apache.hadoop.hdfs.server.datanode.FSDataset.getBlockFile(FSDataset.java:1267)

Three of the top four contention points are on the FSDataset monitor lock. The client-side
DFSInputStream.getFileLength one is surprising, but not related to this particular JIRA.

> Convert FSDataset to ReadWriteLock
> ----------------------------------
>                 Key: HDFS-1148
>                 URL: https://issues.apache.org/jira/browse/HDFS-1148
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hdfs-1148-old.txt, patch-HDFS-1148-rel0.20.2.txt
> In benchmarking HDFS-941 I noticed that for the random read workload, the FSDataset lock
is highly contended. After converting it to a ReentrantReadWriteLock, I saw a ~25% improvement
on both latency and ops/second.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message