I checked out source code from http://svn.apache.org/repos/asf/hadoop/common/branches/branch-2.3 and it builds.

From TestMRJobsWithHistoryService.txt, the test passed.
What led to this test being singled out among the 454 tests ?

Thanks


On Thu, Feb 6, 2014 at 7:26 PM, Christopher Thomas <christophermauricethomas@gmail.com> wrote:
Yes well I tried 2.3, but I have found a number of problems building it. I had to resort to manually applying patches that I found in the bug tracking lists, which did not seem to have made it into all branches. So for the moment I am sticking with 2.2.0 which is advertised as being stable. 

I apologise for the confusion.

Here is the contents of ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt, though perhaps not that illuminating:


-------------------------------------------------------------------------------
Test set: org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService
-------------------------------------------------------------------------------
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.669 sec




On 7 February 2014 14:12, Ted Yu <yuzhihong@gmail.com> wrote:
The output was from hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-output.txt

Can you show us the contents of ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt ?

BTW hadoop 2.3 release candidate is coming up. You may consider paying more attention to hadoop 2.3

Cheers


On Thu, Feb 6, 2014 at 5:33 PM, Christopher Thomas <christophermauricethomas@gmail.com> wrote:
I included the last part of 

hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt 

in the second half of my initial posting, calling it "the output from TestMRJobsWithHistoryService". Sloppy terminology I know, sorry if I wasn't very clear. 

Regards

Chris 


On 7 February 2014 11:53, Ted Yu <yuzhihong@gmail.com> wrote:
There isn't System.exit call in TestMRJobsWithHistoryService.java

What did hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt say ?

Cheers


On Thu, Feb 6, 2014 at 4:41 PM, Christopher Thomas <christophermauricethomas@gmail.com> wrote:
Hi,

I am a complete beginner to Hadoop, trying to build 2.2.0 from source on a Macbook Pro running OS X Mavericks.


I get the following test failure: 

Forking command line: /bin/sh -c cd /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient && /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -jar /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefirebooter1837947962445626736.jar /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire8537321961333093423tmp /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire_1668661782666182940486tmp
Running org.apache.hadoop.mapreduce.v2.TestMROldApiJobs

Results :

Tests run: 454, Failures: 0, Errors: 0, Skipped: 11

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] hadoop-mapreduce-client-jobclient ................. FAILURE [1:18:07.440s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED
[INFO] Apache Hadoop MapReduce Examples .................. SKIPPED
[INFO] hadoop-mapreduce .................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:18:07.828s
[INFO] Finished at: Thu Feb 06 23:01:45 EST 2014
[INFO] Final Memory: 32M/94M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-mapreduce-client-jobclient: ExecutionException; nested exception is java.util.concurrent.ExecutionException: java.lang.RuntimeException: The forked VM terminated without saying properly goodbye. VM crash or System.exit called ?


and the output from TestMRJobsWithHistoryService includes:


2014-02-06 23:00:54,530 ERROR [Thread[Thread-252,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(547)) - InterruptedExcpetion recieved for ExpiredTokenRemover thread java.lang.InterruptedException: sleep interrupted
2014-02-06 23:00:54,531 INFO  [main] ipc.Server (Server.java:stop(2437)) - Stopping server on 55131
2014-02-06 23:00:54,540 INFO  [IPC Server listener on 55131] ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55131
2014-02-06 23:00:54,540 INFO  [IPC Server Responder] ipc.Server (Server.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:00:54,544 INFO  [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped SelectChannelConnector@Christophers-MacBook-Pro.local:0
2014-02-06 23:00:54,647 INFO  [main] hs.JobHistory (JobHistory.java:serviceStop(136)) - Stopping JobHistory
2014-02-06 23:00:54,647 INFO  [main] hs.JobHistory (JobHistory.java:serviceStop(138)) - Stopping History Cleaner/Move To Done
2014-02-06 23:00:54,656 INFO  [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped SelectChannelConnector@Christophers-MacBook-Pro.local:0
2014-02-06 23:00:54,759 INFO  [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(328)) - Applications still running : [application_1391688017341_0001]
2014-02-06 23:00:54,762 INFO  [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(336)) - Waiting for Applications to be Finished
2014-02-06 23:00:54,762 INFO  [AsyncDispatcher event handler] application.Application (ApplicationImpl.java:handle(451)) - Application application_1391688017341_0001 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP
2014-02-06 23:00:54,763 INFO  [DeletionService #3] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute path : /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_0/usercache/root/appcache/application_1391688017341_0001
2014-02-06 23:00:54,763 INFO  [DeletionService #0] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute path : /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_1/usercache/root/appcache/application_1391688017341_0001
2014-02-06 23:00:54,765 INFO  [AsyncDispatcher event handler] containermanager.AuxServices (AuxServices.java:handle(175)) - Got event APPLICATION_STOP for appId application_1391688017341_0001
2014-02-06 23:00:54,765 INFO  [DeletionService #1] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute path : /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_2/usercache/root/appcache/application_1391688017341_0001
2014-02-06 23:00:54,766 INFO  [AsyncDispatcher event handler] application.Application (ApplicationImpl.java:handle(451)) - Application application_1391688017341_0001 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED
2014-02-06 23:00:54,767 INFO  [AsyncDispatcher event handler] loghandler.NonAggregatingLogHandler (NonAggregatingLogHandler.java:handle(118)) - Scheduling Log Deletion for application: application_1391688017341_0001, with delay of 10800 seconds
2014-02-06 23:00:54,768 INFO  [DeletionService #3] nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute path : /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_3/usercache/root/appcache/application_1391688017341_0001
2014-02-06 23:00:58,766 INFO  [main] containermanager.ContainerManagerImpl (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(353)) - Done waiting for Applications to be Finished. Still alive: [application_1391688017341_0001]
2014-02-06 23:00:58,768 INFO  [main] ipc.Server (Server.java:stop(2437)) - Stopping server on 55125
2014-02-06 23:00:58,771 INFO  [IPC Server listener on 55125] ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55125
2014-02-06 23:00:58,772 INFO  [IPC Server Responder] ipc.Server (Server.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:01:08,774 INFO  [main] monitor.ContainersMonitorImpl (ContainersMonitorImpl.java:isEnabled(159)) - ResourceCalculatorPlugin is unavailable on this system. org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl is disabled.
2014-02-06 23:01:08,784 INFO  [main] ipc.Server (Server.java:stop(2437)) - Stopping server on 55126
2014-02-06 23:01:08,785 INFO  [IPC Server listener on 55126] ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55126
2014-02-06 23:01:08,785 INFO  [IPC Server Responder] ipc.Server (Server.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:01:08,785 INFO  [Public Localizer] localizer.ResourceLocalizationService (ResourceLocalizationService.java:run(728)) - Public cache exiting
2014-02-06 23:01:08,787 INFO  [main] mortbay.log (Slf4jLog.java:info(67)) - Stopped SelectChannelConnector@Christophers-MacBook-Pro.local:0
2014-02-06 23:01:08,889 ERROR [Thread[Thread-17,5,main]] delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:run(547)) - InterruptedExcpetion recieved for ExpiredTokenRemover thread java.lang.InterruptedException: sleep interrupted
2014-02-06 23:01:08,891 INFO  [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(200)) - Stopping JobHistoryServer metrics system...
2014-02-06 23:01:08,895 INFO  [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:stop(206)) - JobHistoryServer metrics system stopped.
2014-02-06 23:01:08,895 INFO  [main] impl.MetricsSystemImpl (MetricsSystemImpl.java:shutdown(572)) - JobHistoryServer metrics system shutdown complete.
2014-02-06 23:01:08,896 WARN  [ApplicationMaster Launcher] amlauncher.ApplicationMasterLauncher (ApplicationMasterLauncher.java:run(98)) - org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning.
2014-02-06 23:01:08,897 INFO  [main] ipc.Server (Server.java:stop(2437)) - Stopping server on 55122
2014-02-06 23:01:08,898 INFO  [IPC Server listener on 55122] ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55122
2014-02-06 23:01:08,898 INFO  [main] ipc.Server (Server.java:stop(2437)) - Stopping server on 55121
2014-02-06 23:01:08,898 INFO  [IPC Server Responder] ipc.Server (Server.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:01:08,902 INFO  [IPC Server listener on 55121] ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55121
2014-02-06 23:01:08,908 INFO  [IPC Server Responder] ipc.Server (Server.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:01:08,902 INFO  [main] ipc.Server (Server.java:stop(2437)) - Stopping server on 55120
2014-02-06 23:01:08,912 INFO  [IPC Server listener on 55120] ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55120
2014-02-06 23:01:08,912 INFO  [IPC Server Responder] ipc.Server (Server.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:01:08,912 INFO  [main] ipc.Server (Server.java:stop(2437)) - Stopping server on 55119
2014-02-06 23:01:08,917 INFO  [IPC Server listener on 55119] ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55119
2014-02-06 23:01:08,917 INFO  [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) - NMLivelinessMonitor thread interrupted
2014-02-06 23:01:08,917 ERROR [ResourceManager Event Processor] resourcemanager.ResourceManager (ResourceManager.java:run(435)) - Returning, interrupted : java.lang.InterruptedException
2014-02-06 23:01:08,917 INFO  [IPC Server Responder] ipc.Server (Server.java:run(866)) - Stopping IPC Server Responder
2014-02-06 23:01:08,918 INFO  [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) - AMLivelinessMonitor thread interrupted
2014-02-06 23:01:08,918 INFO  [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) - org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer thread interrupted
2014-02-06 23:01:08,918 INFO  [Ping Checker] util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) - AMLivelinessMonitor thread interrupted

Has anyone seen this error before when building hadoop, and does anyone have any ideas?

Regards

Chris