Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6BF00100C3 for ; Fri, 7 Feb 2014 23:54:18 +0000 (UTC) Received: (qmail 9910 invoked by uid 500); 7 Feb 2014 23:54:10 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 9813 invoked by uid 500); 7 Feb 2014 23:54:09 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 9806 invoked by uid 99); 7 Feb 2014 23:54:09 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Feb 2014 23:54:09 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates 209.85.160.175 as permitted sender) Received: from [209.85.160.175] (HELO mail-yk0-f175.google.com) (209.85.160.175) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Feb 2014 23:54:01 +0000 Received: by mail-yk0-f175.google.com with SMTP id 131so978954ykp.6 for ; Fri, 07 Feb 2014 15:53:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=nFrPByZC0OkAeExL9WeCORUle9oSKzbm+XQ7DaQxp8Y=; b=Hh0Zl9IGwQ41SKzHDGx8oGiUrSab4FhnJ8spA91emONytmvUaQUwPkYNh8cbBfC8K1 mXXI7VrzgjEMdW3k/FLuTH75x2f4wgJ97GL4cb0RV59C2uvfVCbow+s+jdFCvJt/gg3n F9NBSb27wWgOzXBOp/cC4TPukPjaXyT9Gc5Yp1Z/k/jLjCg0MH0joqLKbs69CaALTjJI BO6k+CKSMNku7DwiNGIitieRnQy4FBkF6upPwoaIbWbxMbRLEN2AEeakJzaKVRuLh4qC ZzXSj7rY/i+L5SjMm9IT9bty11vbY88o1Z1veirXwg/pRNtAgi8fZN1JkeRmVxnRNKqg huOA== MIME-Version: 1.0 X-Received: by 10.236.181.137 with SMTP id l9mr2281624yhm.97.1391817220264; Fri, 07 Feb 2014 15:53:40 -0800 (PST) Received: by 10.170.213.65 with HTTP; Fri, 7 Feb 2014 15:53:40 -0800 (PST) In-Reply-To: References: Date: Fri, 7 Feb 2014 15:53:40 -0800 Message-ID: Subject: Re: Problems building hadoop 2.2.0 from source From: Ted Yu To: "common-user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=20cf3040ea100199ca04f1d9b3e7 X-Virus-Checked: Checked by ClamAV on apache.org --20cf3040ea100199ca04f1d9b3e7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable In the output for a passing test, I saw: 2014-02-06 16:48:49,722 ERROR [Thread[Thread-71,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(557)) - InterruptedExcpetion recieved for ExpiredTokenRemover thread java.lang.InterruptedException: sleep interrupted Meaning the above was not critical. branch-2.3 is receiving attention now. Discovering test failure there would be more helpful. Cheers On Thu, Feb 6, 2014 at 9:25 PM, Christopher Thomas < christophermauricethomas@gmail.com> wrote: > I guess the ERROR lines in > > hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client- > jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2. > TestMRJobsWithHistoryService-output.txt > > led me to believe that the problem was with TestMRJobsWithHistoryService. > If that's not the case then what do these messages indicate? > > As I say I am a complete novice and the learning curve is very steep. > > > On 7 February 2014 14:47, Ted Yu wrote: > >> I checked out source code from >> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-2.3 and it >> builds. >> >> From TestMRJobsWithHistoryService.txt, the test passed. >> What led to this test being singled out among the 454 tests ? >> >> Thanks >> >> >> On Thu, Feb 6, 2014 at 7:26 PM, Christopher Thomas < >> christophermauricethomas@gmail.com> wrote: >> >>> Yes well I tried 2.3, but I have found a number of problems building it= . >>> I had to resort to manually applying patches that I found in the bug >>> tracking lists, which did not seem to have made it into all branches. S= o >>> for the moment I am sticking with 2.2.0 which is advertised as being >>> stable. >>> >>> I apologise for the confusion. >>> >>> Here is the contents >>> of ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-= client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.Tes= tMRJobsWithHistoryService.txt, >>> though perhaps not that illuminating: >>> >>> >>> >>> -----------------------------------------------------------------------= -------- >>> Test set: org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService >>> >>> -----------------------------------------------------------------------= -------- >>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.669 >>> sec >>> >>> >>> >>> >>> On 7 February 2014 14:12, Ted Yu wrote: >>> >>>> The output was >>>> from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce= -client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.Te= stMRJobsWithHistoryService-output.txt >>>> >>>> Can you show us the contents >>>> of ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce= -client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.Te= stMRJobsWithHistoryService.txt >>>> ? >>>> >>>> BTW hadoop 2.3 release candidate is coming up. You may consider paying >>>> more attention to hadoop 2.3 >>>> >>>> Cheers >>>> >>>> >>>> On Thu, Feb 6, 2014 at 5:33 PM, Christopher Thomas < >>>> christophermauricethomas@gmail.com> wrote: >>>> >>>>> I included the last part of >>>>> >>>>> hadoop-mapreduce-project/hadoop-mapreduce-client/ >>>>> hadoop-mapreduce-client-jobclient/target/surefire- >>>>> reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService. >>>>> txt >>>>> >>>>> in the second half of my initial posting, calling it "the output from= TestMRJobsWithHistoryService". >>>>> Sloppy terminology I know, sorry if I wasn't very clear. >>>>> >>>>> Regards >>>>> >>>>> Chris >>>>> >>>>> >>>>> On 7 February 2014 11:53, Ted Yu wrote: >>>>> >>>>>> There isn't System.exit call in TestMRJobsWithHistoryService.java >>>>>> >>>>>> What >>>>>> did hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduc= e-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.T= estMRJobsWithHistoryService.txt >>>>>> say ? >>>>>> >>>>>> Cheers >>>>>> >>>>>> >>>>>> On Thu, Feb 6, 2014 at 4:41 PM, Christopher Thomas < >>>>>> christophermauricethomas@gmail.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am a complete beginner to Hadoop, trying to build 2.2.0 from >>>>>>> source on a Macbook Pro running OS X Mavericks. >>>>>>> >>>>>>> I am following the 'instructions' at >>>>>>> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-co= mmon/SingleCluster.html such >>>>>>> as they are. >>>>>>> >>>>>>> I get the following test failure: >>>>>>> >>>>>>> Forking command line: /bin/sh -c cd >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapr= educe-client/hadoop-mapreduce-client-jobclient >>>>>>> && >>>>>>> /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bi= n/java >>>>>>> -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -jar >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapr= educe-client/hadoop-mapreduce-client-jobclient/target/surefire/surefireboot= er1837947962445626736.jar >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapr= educe-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire8537= 321961333093423tmp >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapr= educe-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire_166= 8661782666182940486tmp >>>>>>> Running org.apache.hadoop.mapreduce.v2.TestMROldApiJobs >>>>>>> >>>>>>> Results : >>>>>>> >>>>>>> Tests run: 454, Failures: 0, Errors: 0, Skipped: 11 >>>>>>> >>>>>>> [INFO] >>>>>>> -------------------------------------------------------------------= ----- >>>>>>> [INFO] Reactor Summary: >>>>>>> [INFO] >>>>>>> [INFO] hadoop-mapreduce-client-jobclient ................. FAILURE >>>>>>> [1:18:07.440s] >>>>>>> [INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED >>>>>>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED >>>>>>> [INFO] hadoop-mapreduce .................................. SKIPPED >>>>>>> [INFO] >>>>>>> -------------------------------------------------------------------= ----- >>>>>>> [INFO] BUILD FAILURE >>>>>>> [INFO] >>>>>>> -------------------------------------------------------------------= ----- >>>>>>> [INFO] Total time: 1:18:07.828s >>>>>>> [INFO] Finished at: Thu Feb 06 23:01:45 EST 2014 >>>>>>> [INFO] Final Memory: 32M/94M >>>>>>> [INFO] >>>>>>> -------------------------------------------------------------------= ----- >>>>>>> [ERROR] Failed to execute goal >>>>>>> org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default= -test) >>>>>>> on project hadoop-mapreduce-client-jobclient: ExecutionException; n= ested >>>>>>> exception is java.util.concurrent.ExecutionException: >>>>>>> java.lang.RuntimeException: The forked VM terminated without saying >>>>>>> properly goodbye. VM crash or System.exit called ? >>>>>>> >>>>>>> >>>>>>> and the output from TestMRJobsWithHistoryService includes: >>>>>>> >>>>>>> >>>>>>> 2014-02-06 23:00:54,530 ERROR [Thread[Thread-252,5,main]] >>>>>>> delegation.AbstractDelegationTokenSecretManager >>>>>>> (AbstractDelegationTokenSecretManager.java:run(547)) - InterruptedE= xcpetion >>>>>>> recieved for ExpiredTokenRemover thread java.lang.InterruptedExcept= ion: >>>>>>> sleep interrupted >>>>>>> 2014-02-06 23:00:54,531 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55131 >>>>>>> 2014-02-06 23:00:54,540 INFO [IPC Server listener on 55131] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on= 55131 >>>>>>> 2014-02-06 23:00:54,540 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:00:54,544 INFO [main] mortbay.log >>>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>>> SelectChannelConnector@Christophers-MacBook-Pro.local:0 >>>>>>> 2014-02-06 23:00:54,647 INFO [main] hs.JobHistory >>>>>>> (JobHistory.java:serviceStop(136)) - Stopping JobHistory >>>>>>> 2014-02-06 23:00:54,647 INFO [main] hs.JobHistory >>>>>>> (JobHistory.java:serviceStop(138)) - Stopping History Cleaner/Move = To Done >>>>>>> 2014-02-06 23:00:54,656 INFO [main] mortbay.log >>>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>>> SelectChannelConnector@Christophers-MacBook-Pro.local:0 >>>>>>> 2014-02-06 23:00:54,759 INFO [main] >>>>>>> containermanager.ContainerManagerImpl >>>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(328)) - >>>>>>> Applications still running : [application_1391688017341_0001] >>>>>>> 2014-02-06 23:00:54,762 INFO [main] >>>>>>> containermanager.ContainerManagerImpl >>>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(336)) - = Waiting >>>>>>> for Applications to be Finished >>>>>>> 2014-02-06 23:00:54,762 INFO [AsyncDispatcher event handler] >>>>>>> application.Application (ApplicationImpl.java:handle(451)) - Applic= ation >>>>>>> application_1391688017341_0001 transitioned from RUNNING to >>>>>>> APPLICATION_RESOURCES_CLEANINGUP >>>>>>> 2014-02-06 23:00:54,763 INFO [DeletionService #3] >>>>>>> nodemanager.DefaultContainerExecutor >>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolu= te path >>>>>>> : >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapr= educe-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.map= reduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestM= RJobsWithHistoryService-localDir-nm-0_0/usercache/root/appcache/application= _1391688017341_0001 >>>>>>> 2014-02-06 23:00:54,763 INFO [DeletionService #0] >>>>>>> nodemanager.DefaultContainerExecutor >>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolu= te path >>>>>>> : >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapr= educe-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.map= reduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestM= RJobsWithHistoryService-localDir-nm-0_1/usercache/root/appcache/application= _1391688017341_0001 >>>>>>> 2014-02-06 23:00:54,765 INFO [AsyncDispatcher event handler] >>>>>>> containermanager.AuxServices (AuxServices.java:handle(175)) - Got e= vent >>>>>>> APPLICATION_STOP for appId application_1391688017341_0001 >>>>>>> 2014-02-06 23:00:54,765 INFO [DeletionService #1] >>>>>>> nodemanager.DefaultContainerExecutor >>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolu= te path >>>>>>> : >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapr= educe-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.map= reduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestM= RJobsWithHistoryService-localDir-nm-0_2/usercache/root/appcache/application= _1391688017341_0001 >>>>>>> 2014-02-06 23:00:54,766 INFO [AsyncDispatcher event handler] >>>>>>> application.Application (ApplicationImpl.java:handle(451)) - Applic= ation >>>>>>> application_1391688017341_0001 transitioned from >>>>>>> APPLICATION_RESOURCES_CLEANINGUP to FINISHED >>>>>>> 2014-02-06 23:00:54,767 INFO [AsyncDispatcher event handler] >>>>>>> loghandler.NonAggregatingLogHandler >>>>>>> (NonAggregatingLogHandler.java:handle(118)) - Scheduling Log Deleti= on for >>>>>>> application: application_1391688017341_0001, with delay of 10800 se= conds >>>>>>> 2014-02-06 23:00:54,768 INFO [DeletionService #3] >>>>>>> nodemanager.DefaultContainerExecutor >>>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolu= te path >>>>>>> : >>>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapr= educe-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.map= reduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestM= RJobsWithHistoryService-localDir-nm-0_3/usercache/root/appcache/application= _1391688017341_0001 >>>>>>> 2014-02-06 23:00:58,766 INFO [main] >>>>>>> containermanager.ContainerManagerImpl >>>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(353)) - = Done >>>>>>> waiting for Applications to be Finished. Still alive: >>>>>>> [application_1391688017341_0001] >>>>>>> 2014-02-06 23:00:58,768 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55125 >>>>>>> 2014-02-06 23:00:58,771 INFO [IPC Server listener on 55125] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on= 55125 >>>>>>> 2014-02-06 23:00:58,772 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,774 INFO [main] monitor.ContainersMonitorImpl >>>>>>> (ContainersMonitorImpl.java:isEnabled(159)) - ResourceCalculatorPlu= gin is >>>>>>> unavailable on this system. >>>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.= ContainersMonitorImpl >>>>>>> is disabled. >>>>>>> 2014-02-06 23:01:08,784 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55126 >>>>>>> 2014-02-06 23:01:08,785 INFO [IPC Server listener on 55126] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on= 55126 >>>>>>> 2014-02-06 23:01:08,785 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,785 INFO [Public Localizer] >>>>>>> localizer.ResourceLocalizationService >>>>>>> (ResourceLocalizationService.java:run(728)) - Public cache exiting >>>>>>> 2014-02-06 23:01:08,787 INFO [main] mortbay.log >>>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>>> SelectChannelConnector@Christophers-MacBook-Pro.local:0 >>>>>>> 2014-02-06 23:01:08,889 ERROR [Thread[Thread-17,5,main]] >>>>>>> delegation.AbstractDelegationTokenSecretManager >>>>>>> (AbstractDelegationTokenSecretManager.java:run(547)) - InterruptedE= xcpetion >>>>>>> recieved for ExpiredTokenRemover thread java.lang.InterruptedExcept= ion: >>>>>>> sleep interrupted >>>>>>> 2014-02-06 23:01:08,891 INFO [main] impl.MetricsSystemImpl >>>>>>> (MetricsSystemImpl.java:stop(200)) - Stopping JobHistoryServer metr= ics >>>>>>> system... >>>>>>> 2014-02-06 23:01:08,895 INFO [main] impl.MetricsSystemImpl >>>>>>> (MetricsSystemImpl.java:stop(206)) - JobHistoryServer metrics syste= m >>>>>>> stopped. >>>>>>> 2014-02-06 23:01:08,895 INFO [main] impl.MetricsSystemImpl >>>>>>> (MetricsSystemImpl.java:shutdown(572)) - JobHistoryServer metrics s= ystem >>>>>>> shutdown complete. >>>>>>> 2014-02-06 23:01:08,896 WARN [ApplicationMaster Launcher] >>>>>>> amlauncher.ApplicationMasterLauncher >>>>>>> (ApplicationMasterLauncher.java:run(98)) - >>>>>>> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.Applicatio= nMasterLauncher$LauncherThread >>>>>>> interrupted. Returning. >>>>>>> 2014-02-06 23:01:08,897 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55122 >>>>>>> 2014-02-06 23:01:08,898 INFO [IPC Server listener on 55122] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on= 55122 >>>>>>> 2014-02-06 23:01:08,898 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55121 >>>>>>> 2014-02-06 23:01:08,898 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,902 INFO [IPC Server listener on 55121] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on= 55121 >>>>>>> 2014-02-06 23:01:08,908 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,902 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55120 >>>>>>> 2014-02-06 23:01:08,912 INFO [IPC Server listener on 55120] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on= 55120 >>>>>>> 2014-02-06 23:01:08,912 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,912 INFO [main] ipc.Server >>>>>>> (Server.java:stop(2437)) - Stopping server on 55119 >>>>>>> 2014-02-06 23:01:08,917 INFO [IPC Server listener on 55119] >>>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on= 55119 >>>>>>> 2014-02-06 23:01:08,917 INFO [Ping Checker] >>>>>>> util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(= 127)) - >>>>>>> NMLivelinessMonitor thread interrupted >>>>>>> 2014-02-06 23:01:08,917 ERROR [ResourceManager Event Processor] >>>>>>> resourcemanager.ResourceManager (ResourceManager.java:run(435)) - >>>>>>> Returning, interrupted : java.lang.InterruptedException >>>>>>> 2014-02-06 23:01:08,917 INFO [IPC Server Responder] ipc.Server >>>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>>> util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(= 127)) - >>>>>>> AMLivelinessMonitor thread interrupted >>>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>>> util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(= 127)) - >>>>>>> org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.Container= AllocationExpirer >>>>>>> thread interrupted >>>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>>> util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(= 127)) - >>>>>>> AMLivelinessMonitor thread interrupted >>>>>>> >>>>>>> Has anyone seen this error before when building hadoop, and does >>>>>>> anyone have any ideas? >>>>>>> >>>>>>> Regards >>>>>>> >>>>>>> Chris >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> > --20cf3040ea100199ca04f1d9b3e7 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
In the output for a passing test, I saw:

2014= -02-06 16:48:49,722 ERROR [Thread[Thread-71,5,main]] delegation.AbstractDel= egationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(55= 7)) - =A0 =A0 =A0 =A0 =A0 =A0 InterruptedExcpetion recieved for ExpiredToke= nRemover thread java.lang.InterruptedException: sleep interrupted

Meaning the above was not critical.

branch-2.3 is receiving attention now.
Discover= ing test failure there would be more helpful.

Cheers


On Thu, Feb 6, 2014 at 9:25 PM, Christopher Thomas &= lt;= christophermauricethomas@gmail.com> wrote:
I guess the ERROR lines in=A0<= /div>

hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/tar= get/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-output.txt

led= me to believe that the problem was with=A0TestMRJobsWithHistoryService. If that= 9;s not the case then what do these messages indicate?

As = I say I am a complete novice and the learning curve is very steep.=A0


On 7 Fe= bruary 2014 14:47, Ted Yu <yuzhihong@gmail.com> wrote:
=
I checked out source code from=A0http= ://svn.apache.org/repos/asf/hadoop/common/branches/branch-2.3 and it bu= ilds.

From=A0TestMRJobsWithHistoryService.txt, the test passed.
What led to this test being singled out among the=A0454 tests= ?

Thanks


On Thu, Feb 6, 2014 at 7:26 PM, Christop= her Thomas <christophermauricethomas@gmail.com> wrote:
Yes well I tried 2.3, = but I have found a number of problems building it. I had to resort to manua= lly applying patches that I found in the bug tracking lists, which did not = seem to have made it into all branches. So for the moment I am sticking wit= h 2.2.0 which is advertised as being stable.=A0

I apologise for the confusion.

Here is=A0= the contents of=A0./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop= -mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapre= duce.v2.TestMRJobsWithHistoryService.txt, though perhaps not that illuminat= ing:


-----------------------------------= --------------------------------------------
Test set: org.apache= .hadoop.mapreduce.v2.TestMRJobsWithHistoryService
----------------------------------------------------------------------= ---------
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time = elapsed: 52.669 sec




On 7 February 2014 14:12, Ted Yu <yuz= hihong@gmail.com> wrote:
The output was from=A0hadoop-mapreduce-project/hadoop-mapr= educe-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.= apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-output.txt

Can you show us the contents of=A0./hadoop-mapreduce-project/hadoop-ma= preduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/or= g.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt ?

BTW hadoop 2.3 release candidate is coming up. You may consi= der paying more attention to hadoop 2.3

Cheers


On Th= u, Feb 6, 2014 at 5:33 PM, Christopher Thomas <christophe= rmauricethomas@gmail.com> wrote:
I included the last part of=A0

hadoop-mapre= duce-project/hadoop-map= reduce-client/hadoop-ma= preduce-client-jobclien= t/target/surefire-repor= ts/org.apache.hadoop.ma= preduce.v2.TestMRJobsWi= thHistoryService.txt=A0=

in the second half of my initial posting, calling it &quo= t;the=A0output from=A0TestMRJobsWithHistoryService". S= loppy terminology I know, sorry if I wasn't very clear.=A0

Regards

Chris=A0
=


On 7 February 2014 11:53, Ted Yu <yuz= hihong@gmail.com> wrote:
There isn't=A0System.exit call in=A0TestMRJobsWithHistoryService.java

What did=A0hadoop-mapred= uce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/targe= t/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryServ= ice.txt say ?

Cheers

<= br>
On Thu, Feb 6, 2014 at 4:41 PM, Christopher T= homas <christophermauricethomas@gmail.com> = wrote:
Hi,

I am a= complete beginner to Hadoop, trying to build 2.2.0 from source on a Macboo= k Pro running OS X Mavericks.

I am following the = 9;instructions' at=A0http= ://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleC= luster.html=A0such as they are.

I get the following te= st failure:=A0

F= orking command line: /bin/sh -c cd /Users/hadoop/hadoop-2.2.0-src/hadoop-ma= preduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient &= amp;& /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/= bin/java -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -jar /Users/hadoop/hadoo= p-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapredu= ce-client-jobclient/target/surefire/surefirebooter1837947962445626736.jar /= Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-cli= ent/hadoop-mapreduce-client-jobclient/target/surefire/surefire8537321961333= 093423tmp /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-ma= preduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire_1= 668661782666182940486tmp
Running org.apache.hadoop.mapreduce.v2.TestMROldApiJobs

=
Results :

Tests run: 454, Failures: 0, = Errors: 0, Skipped: 11

[INFO] --------------------= ----------------------------------------------------
[INFO] Reactor Summary:
[INFO]=A0
[INFO] hadoop-ma= preduce-client-jobclient ................. FAILURE [1:18:07.440s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED
[INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
[INFO] hadoop-mapreduce .................................. SKIPPED
[INFO] -----------------------------------------------------------= -------------
[INFO] BUILD FAILURE
[INFO] --------------------------------= ----------------------------------------
[INFO] Total time: 1:18:= 07.828s
[INFO] Finished at: Thu Feb 06 23:01:45 EST 2014
[INFO] Final Memory: 32M/94M
[INFO] ------------------------= ------------------------------------------------
[ERROR] Failed t= o execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (= default-test) on project hadoop-mapreduce-client-jobclient: ExecutionExcept= ion; nested exception is java.util.concurrent.ExecutionException: java.lang= .RuntimeException: The forked VM terminated without saying properly goodbye= . VM crash or System.exit called ?


and the output from=A0TestMRJobsWithHist= oryService includes:


2014-02-0= 6 23:00:54,530 ERROR [Thread[Thread-252,5,main]] delegation.AbstractDelegat= ionTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(547)) = - InterruptedExcpetion recieved for ExpiredTokenRemover thread java.lang.In= terruptedException: sleep interrupted
2014-02-06 23:00:54,531 INFO =A0[main] ipc.Server (Server.java:stop(24= 37)) - Stopping server on 55131
2014-02-06 23:00:54,540 INFO =A0[= IPC Server listener on 55131] ipc.Server (Server.java:run(720)) - Stopping = IPC Server listener on 55131
2014-02-06 23:00:54,540 INFO =A0[IPC Server Responder] ipc.Server (Ser= ver.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:= 00:54,544 INFO =A0[main] mortbay.log (Slf4jLog.java:info(67)) - Stopped Sel= ectChannelConnector@Christophers-MacBook-Pro.local:0
2014-02-06 23:00:54,647 INFO =A0[main] hs.JobHistory (JobHistory.java:= serviceStop(136)) - Stopping JobHistory
2014-02-06 23:00:54,647 I= NFO =A0[main] hs.JobHistory (JobHistory.java:serviceStop(138)) - Stopping H= istory Cleaner/Move To Done
2014-02-06 23:00:54,656 INFO =A0[main] mortbay.log (Slf4jLog.java:info= (67)) - Stopped SelectChannelConnector@Christophers-MacBook-Pro.local:0
2014-02-06 23:00:54,759 INFO =A0[main] containermanager.ContainerMan= agerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(328)) -= Applications still running : [application_1391688017341_0001]
2014-02-06 23:00:54,762 INFO =A0[main] containermanager.ContainerManag= erImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(336)) - W= aiting for Applications to be Finished
2014-02-06 23:00:54,762 IN= FO =A0[AsyncDispatcher event handler] application.Application (ApplicationI= mpl.java:handle(451)) - Application application_1391688017341_0001 transiti= oned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP
2014-02-06 23:00:54,763 INFO =A0[DeletionService #3] nodemanager.Defau= ltContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(368)) - Del= eting absolute path : /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-proje= ct/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apa= che.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapr= educe.v2.TestMRJobsWithHistoryService-localDir-nm-0_0/usercache/root/appcac= he/application_1391688017341_0001
2014-02-06 23:00:54,763 INFO =A0[DeletionService #0] nodemanager.Defau= ltContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(368)) - Del= eting absolute path : /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-proje= ct/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apa= che.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapr= educe.v2.TestMRJobsWithHistoryService-localDir-nm-0_1/usercache/root/appcac= he/application_1391688017341_0001
2014-02-06 23:00:54,765 INFO =A0[AsyncDispatcher event handler] contai= nermanager.AuxServices (AuxServices.java:handle(175)) - Got event APPLICATI= ON_STOP for appId application_1391688017341_0001
2014-02-06 23:00= :54,765 INFO =A0[DeletionService #1] nodemanager.DefaultContainerExecutor (= DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute path := /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-c= lient/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.= v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWi= thHistoryService-localDir-nm-0_2/usercache/root/appcache/application_139168= 8017341_0001
2014-02-06 23:00:54,766 INFO =A0[AsyncDispatcher event handler] applic= ation.Application (ApplicationImpl.java:handle(451)) - Application applicat= ion_1391688017341_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP t= o FINISHED
2014-02-06 23:00:54,767 INFO =A0[AsyncDispatcher event handler] loghan= dler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(118)) -= Scheduling Log Deletion for application: application_1391688017341_0001, w= ith delay of 10800 seconds
2014-02-06 23:00:54,768 INFO =A0[DeletionService #3] nodemanager.Defau= ltContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(368)) - Del= eting absolute path : /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-proje= ct/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apa= che.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapr= educe.v2.TestMRJobsWithHistoryService-localDir-nm-0_3/usercache/root/appcac= he/application_1391688017341_0001
2014-02-06 23:00:58,766 INFO =A0[main] containermanager.ContainerManag= erImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(353)) - D= one waiting for Applications to be Finished. Still alive: [application_1391= 688017341_0001]
2014-02-06 23:00:58,768 INFO =A0[main] ipc.Server (Server.java:stop(24= 37)) - Stopping server on 55125
2014-02-06 23:00:58,771 INFO =A0[= IPC Server listener on 55125] ipc.Server (Server.java:run(720)) - Stopping = IPC Server listener on 55125
2014-02-06 23:00:58,772 INFO =A0[IPC Server Responder] ipc.Server (Ser= ver.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:= 01:08,774 INFO =A0[main] monitor.ContainersMonitorImpl (ContainersMonitorIm= pl.java:isEnabled(159)) - ResourceCalculatorPlugin is unavailable on this s= ystem. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.C= ontainersMonitorImpl is disabled.
2014-02-06 23:01:08,784 INFO =A0[main] ipc.Server (Server.java:stop(24= 37)) - Stopping server on 55126
2014-02-06 23:01:08,785 INFO =A0[= IPC Server listener on 55126] ipc.Server (Server.java:run(720)) - Stopping = IPC Server listener on 55126
2014-02-06 23:01:08,785 INFO =A0[IPC Server Responder] ipc.Server (Ser= ver.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:= 01:08,785 INFO =A0[Public Localizer] localizer.ResourceLocalizationService = (ResourceLocalizationService.java:run(728)) - Public cache exiting
2014-02-06 23:01:08,787 INFO =A0[main] mortbay.log (Slf4jLog.java:info= (67)) - Stopped SelectChannelConnector@Christophers-MacBook-Pro.local:0
2014-02-06 23:01:08,889 ERROR [Thread[Thread-17,5,main]] delegation.= AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.= java:run(547)) - InterruptedExcpetion recieved for ExpiredTokenRemover thre= ad java.lang.InterruptedException: sleep interrupted
2014-02-06 23:01:08,891 INFO =A0[main] impl.MetricsSystemImpl (Metrics= SystemImpl.java:stop(200)) - Stopping JobHistoryServer metrics system...
2014-02-06 23:01:08,895 INFO =A0[main] impl.MetricsSystemImpl (Metr= icsSystemImpl.java:stop(206)) - JobHistoryServer metrics system stopped.
2014-02-06 23:01:08,895 INFO =A0[main] impl.MetricsSystemImpl (Metrics= SystemImpl.java:shutdown(572)) - JobHistoryServer metrics system shutdown c= omplete.
2014-02-06 23:01:08,896 WARN =A0[ApplicationMaster Launc= her] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:r= un(98)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.Applicat= ionMasterLauncher$LauncherThread interrupted. Returning.
2014-02-06 23:01:08,897 INFO =A0[main] ipc.Server (Server.java:stop(24= 37)) - Stopping server on 55122
2014-02-06 23:01:08,898 INFO =A0[= IPC Server listener on 55122] ipc.Server (Server.java:run(720)) - Stopping = IPC Server listener on 55122
2014-02-06 23:01:08,898 INFO =A0[main] ipc.Server (Server.java:stop(24= 37)) - Stopping server on 55121
2014-02-06 23:01:08,898 INFO =A0[= IPC Server Responder] ipc.Server (Server.java:run(866)) - Stopping IPC Serv= er Responder
2014-02-06 23:01:08,902 INFO =A0[IPC Server listener on 55121] ipc.Ser= ver (Server.java:run(720)) - Stopping IPC Server listener on 55121
2014-02-06 23:01:08,908 INFO =A0[IPC Server Responder] ipc.Server (Server= .java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:01:08,902 INFO =A0[main] ipc.Server (Server.java:stop(24= 37)) - Stopping server on 55120
2014-02-06 23:01:08,912 INFO =A0[= IPC Server listener on 55120] ipc.Server (Server.java:run(720)) - Stopping = IPC Server listener on 55120
2014-02-06 23:01:08,912 INFO =A0[IPC Server Responder] ipc.Server (Ser= ver.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:= 01:08,912 INFO =A0[main] ipc.Server (Server.java:stop(2437)) - Stopping ser= ver on 55119
2014-02-06 23:01:08,917 INFO =A0[IPC Server listener on 55119] ipc.Ser= ver (Server.java:run(720)) - Stopping IPC Server listener on 55119
2014-02-06 23:01:08,917 INFO =A0[Ping Checker] util.AbstractLivelinessMon= itor (AbstractLivelinessMonitor.java:run(127)) - NMLivelinessMonitor thread= interrupted
2014-02-06 23:01:08,917 ERROR [ResourceManager Event Processor] resour= cemanager.ResourceManager (ResourceManager.java:run(435)) - Returning, inte= rrupted : java.lang.InterruptedException
2014-02-06 23:01:08,917 = INFO =A0[IPC Server Responder] ipc.Server (Server.java:run(866)) - Stopping= IPC Server Responder
2014-02-06 23:01:08,918 INFO =A0[Ping Checker] util.AbstractLiveliness= Monitor (AbstractLivelinessMonitor.java:run(127)) - AMLivelinessMonitor thr= ead interrupted
2014-02-06 23:01:08,918 INFO =A0[Ping Checker] ut= il.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) - or= g.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocation= Expirer thread interrupted
2014-02-06 23:01:08,918 INFO =A0[Ping Checker] util.AbstractLiveliness= Monitor (AbstractLivelinessMonitor.java:run(127)) - AMLivelinessMonitor thr= ead interrupted

Has anyone seen this error before = when building hadoop, and does anyone have any ideas?

Regards

Chris=A0







--20cf3040ea100199ca04f1d9b3e7--