Mailing-List: contact user-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@giraph.apache.org
Received-SPF: pass (athena.apache.org: domain of lawrence.compton@gmail.com
 designates 74.125.82.174 as permitted sender)
MIME-Version: 1.0
Date: Fri, 22 Nov 2013 12:15:14 -0500
Message-ID: 
 <CAK6w7bCv5onmc61OqJFp162r=ExfWqJ5XLeeRqTXXL6t8Wj5eQ@mail.gmail.com>
Subject: Giraph 1.0.0 - Netty port allocation
From: Larry Compton <lawrence.compton@gmail.com>
To: user@giraph.apache.org
Content-Type: multipart/alternative; boundary=047d7b8736a05bcbb104ebc7283d

--047d7b8736a05bcbb104ebc7283d
Content-Type: text/plain; charset=ISO-8859-1

My teammates and I are running Giraph on a cluster where a firewall is
configured on each compute node. We had 100 ports opened on the compute
nodes, which we thought would be more than enough to accommodate a large
number of workers. However, we're unable to go beyond about 90 workers with
our Giraph jobs, due to Netty ports being allocated outside of the range
(30000-30100). We're not sure why this is happening. We shouldn't be
running more than one worker per compute node, so we were assuming that
only port 30000 would be used, but we're routinely seeing Giraph try to use
ports greater than 30100 when we request close to 100 workers. This leads
us to believe that a simple one up numbering scheme is being used that
doesn't take the host into consideration, although this is only speculation.

Is there a way around this problem? Our system admins understandably balked
at opening 1000 ports.

Larry

--047d7b8736a05bcbb104ebc7283d
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>My teammates and I are running Giraph on a cluster wh=
ere a firewall is configured on each compute node. We had 100 ports opened =
on the compute nodes, which we thought would be more than enough to accommo=
date a large number of workers. However, we&#39;re unable to go beyond abou=
t 90 workers with our Giraph jobs, due to Netty ports being allocated outsi=
de of the range (30000-30100). We&#39;re not sure why this is happening. We=
 shouldn&#39;t be running more than one worker per compute node, so we were=
 assuming that only port 30000 would be used, but we&#39;re routinely seein=
g Giraph try to use ports greater than 30100 when we request close to 100 w=
orkers. This leads us to believe that a simple one up numbering scheme is b=
eing used that doesn&#39;t take the host into consideration, although this =
is only speculation.<br>
<br></div><div>Is there a way around this problem? Our system admins unders=
tandably balked at opening 1000 ports.<br><br></div><div>Larry<br></div></d=
iv>

--047d7b8736a05bcbb104ebc7283d--