hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prashant Kommireddi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-980) Nodemanager is shutting down while executing a mapreduce job
Date Sat, 27 Jul 2013 00:15:49 GMT

    [ https://issues.apache.org/jira/browse/YARN-980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13721418#comment-13721418
] 

Prashant Kommireddi commented on YARN-980:
------------------------------------------

[~vinodkv] regarding
{quote}
Didn't know this before, the default number of parallel downloads is 4, can you increase yarn.nodemanager.localizer.fetch.thread-count
and try again?
{quote}

What is a good way to determine the ideal value for this config?

Question for everyone - what is the reasoning behind killing a process when a threshold is
met instead of throttling or something else to that effect? It makes sense in a few cases,
but killing in this case seems quite drastic, no?

                
> Nodemanager is shutting down while executing a mapreduce job
> ------------------------------------------------------------
>
>                 Key: YARN-980
>                 URL: https://issues.apache.org/jira/browse/YARN-980
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Raghu C Doppalapudi
>            Assignee: Vinod Kumar Vavilapalli
>            Priority: Critical
>
> 2013-07-24 11:00:26,582 FATAL event.AsyncDispatcher - Error in dispatcher thread
> java.util.concurrent.RejectedExecutionException
> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768)
> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:152)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$PublicLocalizer.addResource(ResourceLocalizationService.java:621)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:516)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.handle(ResourceLocalizationService.java:458)
> at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:128)
> at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:77)
> at java.lang.Thread.run(Thread.java:662)
> 2013-07-24 11:00:26,582 INFO event.AsyncDispatcher - Exiting, bbye..
> 2013-07-24 11:00:26,583 INFO service.AbstractService - Service:Dispatcher is stopped.
> 2013-07-24 11:00:26,585 INFO mortbay.log - Stopped SelectChannelConnector@0.0.0.0:8042
> 2013-07-24 11:00:26,686 INFO service.AbstractService - Service:org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer
is stopped.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message