hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13113) Enable parallel test execution for hadoop-aws.
Date Mon, 16 May 2016 15:08:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15284700#comment-15284700
] 

Steve Loughran commented on HADOOP-13113:
-----------------------------------------

I should warn that I'm seeing errors here; I think inconsistencies may be surfing —but I
don't see why that should happen with parallel tests. 

Maybe if writes are taking time in the background after a close(), the longer that write takes
to complete, the more chance the next operation may fail —and with multiple threads trying
to use the same network connection, that's generating the load

> Enable parallel test execution for hadoop-aws.
> ----------------------------------------------
>
>                 Key: HADOOP-13113
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13113
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: test
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>             Fix For: 2.8.0
>
>         Attachments: HADOOP-13113.001.patch, HADOOP-13113.002.patch, HADOOP-13113.003.patch,
HADOOP-13113.004.patch
>
>
> The full hadoop-aws test suite takes ~30 minutes to execute.  The tests spend most of
their time blocked on network I/O with the S3 back-end, but they don't saturate the bandwidth
of the NIC.  We can improve overall execution time by enabling parallel test execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message