Return-Path: X-Original-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 44C841033C for ; Fri, 19 Jul 2013 04:51:07 +0000 (UTC) Received: (qmail 49295 invoked by uid 500); 19 Jul 2013 04:51:06 -0000 Delivered-To: apmail-hadoop-mapreduce-issues-archive@hadoop.apache.org Received: (qmail 48945 invoked by uid 500); 19 Jul 2013 04:50:55 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-issues@hadoop.apache.org Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 48932 invoked by uid 99); 19 Jul 2013 04:50:49 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 19 Jul 2013 04:50:49 +0000 Date: Fri, 19 Jul 2013 04:50:49 +0000 (UTC) From: "Tsuyoshi OZAWA (JIRA)" To: mapreduce-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (MAPREDUCE-5404) MAPREDUCE-5265 should have used ephemeral port MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/MAPREDUCE-5404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13713338#comment-13713338 ] Tsuyoshi OZAWA commented on MAPREDUCE-5404: ------------------------------------------- Thanks for your reporting, Ted. I want to make sure the branch name you are using, because the code in branch-2 are implemented to use ephemeral ports if JHAdminConfig.MR_HISTORY_MINICLUSTER_FIXED_PORTS is false as follows: {code:java} private class JobHistoryServerWrapper extends AbstractService { @Override public synchronized void serviceStart() throws Exception { try { if (!getConfig().getBoolean( JHAdminConfig.MR_HISTORY_MINICLUSTER_FIXED_PORTS, JHAdminConfig.DEFAULT_MR_HISTORY_MINICLUSTER_FIXED_PORTS)) { // pick free random ports. getConfig().set(JHAdminConfig.MR_HISTORY_ADDRESS, MiniYARNCluster.getHostname() + ":0"); getConfig().set(JHAdminConfig.MR_HISTORY_WEBAPP_ADDRESS, MiniYARNCluster.getHostname() + ":0"); } historyServer = new JobHistoryServer(); historyServer.init(getConfig()); ... } } {code} > MAPREDUCE-5265 should have used ephemeral port > ---------------------------------------------- > > Key: MAPREDUCE-5404 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5404 > Project: Hadoop Map/Reduce > Issue Type: Bug > Reporter: Ted Yu > > I ran HBase trunk tests against 2.2.0-SNAPSHOT and many mapreduce jobs failed. > Here is one example: > {code} > org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2 Time elapsed: 0.001 sec <<< ERROR! > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.service.ServiceStateException: java.net.BindException: Problem binding to [0.0.0.0:10033] java.net.BindException: Address already in use; For more details see: http://wiki.apache.org/hadoop/BindException > at org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster$JobHistoryServerWrapper.serviceStart(MiniMRYarnCluster.java:177) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:101) > at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:80) > at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:183) > at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:171) > at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:163) > at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:124) > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1751) > at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:1692) > at org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScanBase.setUpBeforeClass(TestTableInputFormatScanBase.java:84) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) > at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at org.junit.runners.Suite.runChild(Suite.java:127) > at org.junit.runners.Suite.runChild(Suite.java:26) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:724) > Caused by: org.apache.hadoop.service.ServiceStateException: java.net.BindException: Problem binding to [0.0.0.0:10033] java.net.BindException: Address already in use; For more details see: http://wiki.apache.org/hadoop/BindException > at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59) > at org.apache.hadoop.service.AbstractService.init(AbstractService.java:172) > at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:88) > at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:91) > at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster$JobHistoryServerWrapper.serviceStart(MiniMRYarnCluster.java:161) > ... 30 more > Caused by: java.net.BindException: Problem binding to [0.0.0.0:10033] java.net.BindException: Address already in use; For more details see: http://wiki.apache.org/hadoop/BindException > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:719) > at org.apache.hadoop.ipc.Server.bind(Server.java:398) > at org.apache.hadoop.ipc.Server$Listener.(Server.java:501) > at org.apache.hadoop.ipc.Server.(Server.java:2168) > at org.apache.hadoop.ipc.RPC$Server.(RPC.java:997) > at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.(ProtobufRpcEngine.java:505) > at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:480) > at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:840) > at org.apache.hadoop.mapreduce.v2.hs.server.HSAdminServer.serviceInit(HSAdminServer.java:93) > at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > ... 34 more > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira