Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 54117B5C4 for ; Wed, 4 Jan 2012 07:44:21 +0000 (UTC) Received: (qmail 3873 invoked by uid 500); 4 Jan 2012 07:44:20 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 3090 invoked by uid 500); 4 Jan 2012 07:43:58 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 3081 invoked by uid 99); 4 Jan 2012 07:43:52 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Jan 2012 07:43:52 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.210.176] (HELO mail-iy0-f176.google.com) (209.85.210.176) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Jan 2012 07:43:44 +0000 Received: by iapp10 with SMTP id p10so43098235iap.35 for ; Tue, 03 Jan 2012 23:43:24 -0800 (PST) Received: by 10.42.72.135 with SMTP id o7mr58126039icj.45.1325663004104; Tue, 03 Jan 2012 23:43:24 -0800 (PST) Received: from [10.0.1.5] (c-98-234-189-94.hsd1.ca.comcast.net. [98.234.189.94]) by mx.google.com with ESMTPS id lu10sm94045828igc.0.2012.01.03.23.42.57 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 03 Jan 2012 23:43:23 -0800 (PST) From: Arun C Murthy Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: multipart/alternative; boundary=Apple-Mail-83--536596941 Subject: Re: Exception from Yarn Launch Container Date: Tue, 3 Jan 2012 23:42:33 -0800 In-Reply-To: To: mapreduce-user@hadoop.apache.org References: Message-Id: X-Mailer: Apple Mail (2.1084) --Apple-Mail-83--536596941 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=GB2312 Bing, Are you using the released version of hadoop-0.23? If so, you might = want to upgrade to latest build off branch-0.23 (i.e. = hadoop-0.23.1-SNAPSHOT) which has the fix for MAPREDUCE-3537. Arun On Dec 29, 2011, at 12:27 AM, Bing Jiang wrote: > Hi, I use Yarn as resource management to deploy my run-time computing = system. I follow =20 >> = http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/= YARN.html >> = http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/= WritingYarnApplications.html > as guide, and I find these issues below.=20 >=20 > yarn-nodemanager-**.log: > .... > 2011-12-29 15:49:16,250 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.application.App= lication: Adding container_1325062142731_0006_01_000001 to application = application_1325062142731_0006 > 2011-12-29 15:49:16,250 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event= .ApplicationLocalizationEvent.EventType: INIT_APPLICATION_RESOURCES > 2011-12-29 15:49:16,250 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.application.App= licationInitedEvent.EventType: APPLICATION_INITED > 2011-12-29 15:49:16,250 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.application.App= lication: Processing application_1325062142731_0006 of type = APPLICATION_INITED > 2011-12-29 15:49:16,250 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.application.App= lication: Application application_1325062142731_0006 transitioned from = INITING to RUNNING > 2011-12-29 15:49:16,250 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.even= t.LogHandlerAppStartedEvent.EventType: APPLICATION_STARTED > 2011-12-29 15:49:16,250 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= inerInitEvent.EventType: INIT_CONTAINER > 2011-12-29 15:49:16,250 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Processing container_1325062142731_0006_01_000001 of type = INIT_CONTAINER > 2011-12-29 15:49:16,250 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Container container_1325062142731_0006_01_000001 transitioned from = NEW to LOCALIZED > 2011-12-29 15:49:16,250 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nersLauncherEvent.EventType: LAUNCH_CONTAINER > 2011-12-29 15:49:16,287 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= inerEvent.EventType: CONTAINER_LAUNCHED > 2011-12-29 15:49:16,287 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Processing container_1325062142731_0006_01_000001 of type = CONTAINER_LAUNCHED > 2011-12-29 15:49:16,287 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Container container_1325062142731_0006_01_000001 transitioned from = LOCALIZED to RUNNING > 2011-12-29 15:49:16,288 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.Contain= erStartMonitoringEvent.EventType: START_MONITORING_CONTAINER > 2011-12-29 15:49:16,289 WARN = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch: Failed to launch container > java.io.FileNotFoundException: File = /tmp/nm-local-dir/usercache/jiangbing/appcache/application_1325062142731_0= 006 does not exist > at = org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.j= ava:431) > at = org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:815) > at = org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:= 143) > at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:189) > at org.apache.hadoop.fs.FileContext$4.next(FileContext.java:700) > at org.apache.hadoop.fs.FileContext$4.next(FileContext.java:697) > at = org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2= 325) > at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:697) > at = org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchC= ontainer(DefaultContainerExecutor.java:123) > at = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch.call(ContainerLaunch.java:237) > at = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch.call(ContainerLaunch.java:67) > at = java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at = java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.= java:886) > at = java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java= :908) > at java.lang.Thread.run(Thread.java:662) > 2011-12-29 15:49:16,290 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= inerExitEvent.EventType: CONTAINER_EXITED_WITH_FAILURE > 2011-12-29 15:49:16,290 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Processing container_1325062142731_0006_01_000001 of type = CONTAINER_EXITED_WITH_FAILURE > 2011-12-29 15:49:16,290 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Container container_1325062142731_0006_01_000001 transitioned from = RUNNING to EXITED_WITH_FAILURE > 2011-12-29 15:49:16,290 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nersLauncherEvent.EventType: CLEANUP_CONTAINER > 2011-12-29 15:49:16,290 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch: Cleaning up container container_1325062142731_0006_01_000001 > 2011-12-29 15:49:16,290 DEBUG = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch: Marking container container_1325062142731_0006_01_000001 as = inactive > 2011-12-29 15:49:16,290 DEBUG = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch: Getting pid for container = container_1325062142731_0006_01_000001 to kill from pid file = /tmp/nm-local-dir/nmPrivate/container_1325062142731_0006_01_000001.pid > 2011-12-29 15:49:16,290 DEBUG = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch: Accessing pid for container = container_1325062142731_0006_01_000001 from pid file = /tmp/nm-local-dir/nmPrivate/container_1325062142731_0006_01_000001.pid > 2011-12-29 15:49:16,307 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event= .ContainerLocalizationCleanupEvent.EventType: = CLEANUP_CONTAINER_RESOURCES >=20 >=20 >=20 > --=20 > Bing Jiang > Tel=A3=BA(86)134-2619-1361 > National Research Center for Intelligent Computing Systems > Institute of Computing technology > Graduate University of Chinese Academy of Science >=20 --Apple-Mail-83--536596941 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=GB2312
Hi, I use = Yarn as resource management to deploy my run-time computing system. I = follow 
as guide, = and I find these issues below. =

yarn-nodemanager-**.log:
....
2011-12-29 15:49:16,250 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.application.App= lication: Adding container_1325062142731_0006_01_000001 to application = application_1325062142731_0006
2011-12-29 15:49:16,250 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event= .ApplicationLocalizationEvent.EventType: INIT_APPLICATION_RESOURCES
2011-12-29 15:49:16,250 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.application.App= licationInitedEvent.EventType: APPLICATION_INITED
2011-12-29 15:49:16,250 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.application.App= lication: Processing application_1325062142731_0006 of type = APPLICATION_INITED
2011-12-29 15:49:16,250 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.application.App= lication: Application application_1325062142731_0006 transitioned from = INITING to RUNNING
2011-12-29 15:49:16,250 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.even= t.LogHandlerAppStartedEvent.EventType: APPLICATION_STARTED
2011-12-29 15:49:16,250 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= inerInitEvent.EventType: INIT_CONTAINER
2011-12-29 15:49:16,250 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Processing container_1325062142731_0006_01_000001 of type = INIT_CONTAINER
2011-12-29 15:49:16,250 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Container container_1325062142731_0006_01_000001 transitioned from = NEW to LOCALIZED
2011-12-29 15:49:16,250 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nersLauncherEvent.EventType: LAUNCH_CONTAINER
2011-12-29 15:49:16,287 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= inerEvent.EventType: CONTAINER_LAUNCHED
2011-12-29 15:49:16,287 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Processing container_1325062142731_0006_01_000001 of type = CONTAINER_LAUNCHED
2011-12-29 15:49:16,287 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Container container_1325062142731_0006_01_000001 transitioned from = LOCALIZED to RUNNING
2011-12-29 15:49:16,288 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.Contain= erStartMonitoringEvent.EventType: START_MONITORING_CONTAINER
2011-12-29 15:49:16,289 WARN = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch: Failed to launch container
java.io.FileNotFoundException: = File = /tmp/nm-local-dir/usercache/jiangbing/appcache/application_1325062142731_0= 006 does not exist
    at = org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.j= ava:431)
    at = org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:815)
&nb= sp;   at = org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:= 143)
    at = org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:189)
  &nbs= p; at = org.apache.hadoop.fs.FileContext$4.next(FileContext.java:700)
 &nb= sp;  at = org.apache.hadoop.fs.FileContext$4.next(FileContext.java:697)
 &nb= sp; at = org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2= 325)
    at = org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:697)
 &nbs= p;  at = org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchC= ontainer(DefaultContainerExecutor.java:123)
    at = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch.call(ContainerLaunch.java:237)
    at = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch.call(ContainerLaunch.java:67)
    at = java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
&nbs= p;   at = java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at = java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.= java:886)
    at = java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java= :908)
    at java.lang.Thread.run(Thread.java:662)
2011-12-29 15:49:16,290 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= inerExitEvent.EventType: CONTAINER_EXITED_WITH_FAILURE
2011-12-29 15:49:16,290 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Processing container_1325062142731_0006_01_000001 of type = CONTAINER_EXITED_WITH_FAILURE
2011-12-29 15:49:16,290 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Conta= iner: Container container_1325062142731_0006_01_000001 transitioned from = RUNNING to EXITED_WITH_FAILURE
2011-12-29 15:49:16,290 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nersLauncherEvent.EventType: CLEANUP_CONTAINER
2011-12-29 15:49:16,290 INFO = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch: Cleaning up container = container_1325062142731_0006_01_000001
2011-12-29 15:49:16,290 DEBUG = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch: Marking container container_1325062142731_0006_01_000001 as = inactive
2011-12-29 15:49:16,290 DEBUG = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch: Getting pid for container = container_1325062142731_0006_01_000001 to kill from pid file = /tmp/nm-local-dir/nmPrivate/container_1325062142731_0006_01_000001.pid
= 2011-12-29 15:49:16,290 DEBUG = org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai= nerLaunch: Accessing pid for container = container_1325062142731_0006_01_000001 from pid file = /tmp/nm-local-dir/nmPrivate/container_1325062142731_0006_01_000001.pid
= 2011-12-29 15:49:16,307 DEBUG = org.apache.hadoop.yarn.event.AsyncDispatcher: Dispatching the event = org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.event= .ContainerLocalizationCleanupEvent.EventType: = CLEANUP_CONTAINER_RESOURCES



--
Bing = Jiang
Tel=A3=BA(86)134-2619-1361
National Research Center for = Intelligent Computing Systems
Institute of Computing = technology
Graduate University of Chinese Academy of Science


= --Apple-Mail-83--536596941--