hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharath Mundlapudi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7255) Performance regression bug caused by locking code
Date Tue, 03 May 2011 05:45:03 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028039#comment-13028039
] 

Bharath Mundlapudi commented on HADOOP-7255:
--------------------------------------------

Thanks for posting performance data. Can you please let us know the test setup details? Like
how big is your cluster?
It is interesting to see high Std Deviation without the patch. Did you try multiple runs?
  


> Performance regression bug caused by locking code
> -------------------------------------------------
>
>                 Key: HADOOP-7255
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7255
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 0.20.203.0
>            Reporter: T Jake Luciani
>
> While testing the 0.20.203 branch we have discovered quite a severe performance regression
caused by the following checkin:
> http://svn.apache.org/viewvc?view=revision&revision=1079260
> This locking seems to cause a pause of a few seconds between tasks, example with DFSIOTest:
> *CURRENT CODE*
> {noformat}
> 11/05/02 21:16:28 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
> 11/05/02 21:16:28 INFO fs.TestDFSIO:            Date & time: Mon May 02 21:16:28
EDT 2011
> 11/05/02 21:16:28 INFO fs.TestDFSIO:        Number of files: 5
> 11/05/02 21:16:28 INFO fs.TestDFSIO: Total MBytes processed: 500
> 11/05/02 21:16:28 INFO fs.TestDFSIO:      Throughput mb/sec: 303.3980582524272
> 11/05/02 21:16:28 INFO fs.TestDFSIO: Average IO rate mb/sec: 327.69122314453125
> 11/05/02 21:16:28 INFO fs.TestDFSIO:  IO rate std deviation: 78.95767116969083
> 11/05/02 21:16:28 INFO fs.TestDFSIO:     Test exec time sec: 19.476
> 11/05/02 21:16:28 INFO fs.TestDFSIO: 
> {noformat}
> *WITH PATCH REMOVED*
> {noformat}
> 11/05/02 21:19:03 INFO fs.TestDFSIO: ----- TestDFSIO ----- : read
> 11/05/02 21:19:03 INFO fs.TestDFSIO:            Date & time: Mon May 02 21:19:03
EDT 2011
> 11/05/02 21:19:03 INFO fs.TestDFSIO:        Number of files: 5
> 11/05/02 21:19:03 INFO fs.TestDFSIO: Total MBytes processed: 500
> 11/05/02 21:19:03 INFO fs.TestDFSIO:      Throughput mb/sec: 366.03221083455344
> 11/05/02 21:19:03 INFO fs.TestDFSIO: Average IO rate mb/sec: 366.35528564453125
> 11/05/02 21:19:03 INFO fs.TestDFSIO:  IO rate std deviation: 11.020931463080379
> 11/05/02 21:19:03 INFO fs.TestDFSIO:     Test exec time sec: 3.543
> 11/05/02 21:19:03 INFO fs.TestDFSIO: 
> {noformat}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message