hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6473) Job submission can take a long time during Cluster initialization
Date Tue, 12 Jan 2016 16:51:40 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094249#comment-15094249
] 

Eric Payne commented on MAPREDUCE-6473:
---------------------------------------

bq. In the above scenario when thread 2 comes along and finds that the provider list is null
, since thread 1 is already in the synchronized block, thread 2 will wait
[~kshukla], Yes, you are correct. I had missed that. Thanks for the clarification.

+1 (non-binding)

> Job submission can take a long time during Cluster initialization
> -----------------------------------------------------------------
>
>                 Key: MAPREDUCE-6473
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6473
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: performance
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>         Attachments: 99%ile.png, MAPREDUCE-6473-PerfTest.txt, MAPREDUCE-6473-v1.patch,
MAPREDUCE-6473-v2.patch, MAPREDUCE-6473-v3.patch, MAPREDUCE-6473-v4.patch, MAPREDUCE-6473-v5.patch,
MAPREDUCE-6473-v6.patch, MAPREDUCE-6473-v7.patch, avgtime.png
>
>
> During initialization in Cluster.java, the framework provider classes are loaded inside
a sync block which can considerably increase job submission time when the number of submissions
are high. The motive is to reduce time spent in this sync block safely to improve performance.
> {noformat}
> synchronized (frameworkLoader) {
>       for (ClientProtocolProvider provider : frameworkLoader) {
>         LOG.debug("Trying ClientProtocolProvider : "
>             + provider.getClass().getName());
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message