Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A9F1410E26 for ; Thu, 22 Oct 2015 18:04:54 +0000 (UTC) Received: (qmail 30871 invoked by uid 500); 22 Oct 2015 18:04:50 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 30768 invoked by uid 500); 22 Oct 2015 18:04:50 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 30758 invoked by uid 99); 22 Oct 2015 18:04:50 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 22 Oct 2015 18:04:50 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id EFDA2C6108 for ; Thu, 22 Oct 2015 18:04:49 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 4.901 X-Spam-Level: **** X-Spam-Status: No, score=4.901 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, KAM_BADIPHTTP=2, NORMAL_HTTP_TO_IP=0.001, SPF_PASS=-0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id NfaYDBvYQOaI for ; Thu, 22 Oct 2015 18:04:34 +0000 (UTC) Received: from mail-lf0-f53.google.com (mail-lf0-f53.google.com [209.85.215.53]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 36EBF439C2 for ; Thu, 22 Oct 2015 18:04:34 +0000 (UTC) Received: by lfaz124 with SMTP id z124so57552518lfa.1 for ; Thu, 22 Oct 2015 11:04:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=MUn9Uyd5U1axL9x+j5KOIV5wnxyrRAb6WZLorQucEMM=; b=WrgNHPV7TiAuo2IsJDv8pvS+kGLaJMb7/wjQPXBmGWzPa7U1R0OO63BmwvWmWJSbcL cOYDsBQnc6BZJmi/TOyNu2kMM+oPVS02Nh+Zh6AkzsV9O3sakBZx8Vr2XCHjv8HHjYtG oh+y++jHVgIMeiA9eZ6QqHBtM2tu+ZO8R7CedsRucTMn8qYzIWQDyivuPs10BxhbExU5 Ue0mMnYIj9odmR9+8z0aMdIIDkgLFWzY7hb8Pe5IvgpiTTtbXxHfWNdIInOfsSKx8rRm FQ29MOXe0BQ0TAlhNP5uYvRk9rbXsRmcYzxgm8w5JuLhfkepKVhbbE9Fv1Y8U6rJ41NZ jbpw== MIME-Version: 1.0 X-Received: by 10.25.18.39 with SMTP id h39mr6043842lfi.7.1445537073080; Thu, 22 Oct 2015 11:04:33 -0700 (PDT) Received: by 10.25.197.198 with HTTP; Thu, 22 Oct 2015 11:04:33 -0700 (PDT) Date: Thu, 22 Oct 2015 14:04:33 -0400 Message-ID: Subject: Two map reduce jobs running at once creates port conflict. From: Edward Capriolo To: "common-user@hadoop.apache.org" Content-Type: multipart/alternative; boundary=001a113fc322c032090522b553d5 --001a113fc322c032090522b553d5 Content-Type: text/plain; charset=UTF-8 I have just updated to CDH 5.4.2. When multiple map reduce jobs run at once a port bind conflict sometimes happens. It seems like from the message that binding to 0.0.0.0:0 will pick a random port which should not cause a conflict but that does not seem to happen. at sun.nio.ch.Net.bind(Net.java:436) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.apache.hadoop.ipc.Server.bind(Server.java:407) ... 19 more 2015-10-04 19:31:10,567 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [0.0.0.0:0] java.net.BindException: Address already in use; For more details see: http://wiki.apache.org/hadoop/BindException org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [0.0.0.0:0] java.net.BindException: Address already in use; For more details see: http://wiki.apache.org/hadoop/BindException at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139) at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnProtoRPC.java:65) at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:119) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1084) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1500) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1496) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1429) Caused by: java.net.BindException: Problem binding to [0.0.0.0:0] java.net.BindException: Address already in use; For more details see: http://wiki.apache.org/hadoop/BindException Does anyone know why this happens? Also a work around that does not involve an upgrade? TX --001a113fc322c032090522b553d5 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable 
I have just updated to CDH 5.4.2.=
=C2=A0

When multiple map reduce jobs run at once a port bind =
conflict sometimes happens. It seems like from the message that binding to =
0.0.0.0:0 will pick a random port which sh=
ould not cause a conflict but that does not seem to happen.
at sun.nio.ch.Net.bind(Net.java:436)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
	at sun.nio.ch.ServerSocketAdaptor.bind(=
ServerSocketAdaptor.java:74)
	at org.apache.hadoop.ipc.Server.bind=
(Server.java:407)
	... 19 more
2015-10-04 19:31:10,567 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapredu=
ce.v2.app.MRAppMaster failed in state=
 STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: j=
ava.net.BindException: Problem binding to [0.0.0.0:=
0] java.net.BindException: Address alrea=
dy in use; For more details see:  http://wiki.apache.org/hadoop/BindException
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindExcep=
tion: Problem binding to [0.0.0.0:0] java.net.BindException: Address already in use; For m=
ore details see:  http://=
wiki.apache.org/hadoop/BindException
	at org.apache.hadoop.yarn.factories.=
impl.pb.RpcServerFactoryPBImpl.getServer(RpcServerFactoryPBImpl.java:139)
	at org.apache.hadoop.yarn.ipc.Hadoop=
YarnProtoRPC.getServer(HadoopYarnProt=
oRPC.java:65)
	at org.apache.hadoop.mapreduce.v2.ap=
p.client.MRClientService.serviceStart=
(MRClientService.java:119)
	at org.apache.hadoop.service.Abstrac=
tService.start(AbstractService.java:1=
93)
	at org.apache.hadoop.mapreduce.v2.ap=
p.MRAppMaster.serviceStart(MRAppMaste=
r.java:1084)
	at org.apache.hadoop.service.Abstrac=
tService.start(AbstractService.java:1=
93)
	at org.apache.hadoop.mapreduce.v2.ap=
p.MRAppMaster$4.run(MRAppMaster.java:=
1500)
	at java.security.AccessController.do=
Privileged(Native Method)
	at javax.security.auth.Subject.doAs(=
Subject.java:415)
	at org.apache.hadoop.security.UserGr=
oupInformation.doAs(UserGroupInformat=
ion.java:1671)
	at org.apache.hadoop.mapreduce.v2.ap=
p.MRAppMaster.initAndStartAppMaster(<=
/span>MRAppMaster.java:1496)
	at org.apache.hadoop.mapreduce.v2.ap=
p.MRAppMaster.main(MRAppMaster.java:1=
429)
Caused by: java.net.BindException: Problem binding to [0.0.0.0:0] java.net.BindException: A=
ddress already in use; For more details see:  http://wiki.apache.org/hadoop/BindException

Does anyone know why this happens? Also a work around that does not involve=
 an upgrade?

TX
--001a113fc322c032090522b553d5--