reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julia Wang (QIUHE)" <Qiuhe.W...@microsoft.com>
Subject RE: Question about Reef-webserver
Date Wed, 08 Oct 2014 16:50:17 GMT
That's right. To be more specific, 

1. Port number in Reef HttpServer is a named parameter that can be bound by client. Typical
example of the usage is 
    final Configuration httpServerConfiguration = Tang.Factory.getTang().newConfigurationBuilder()
        .bindNamedParameter(PortNumber.class, "9000")
        .build();
If you don't bind, the default is 8080. We can change the class to be public if that prevents
external user to bind. 

2. However, We cannot guarantee this port number is available when running on a machine. If
there is a conflict, we will generate one on the fly within the given range. Client can specify
this range by binding MaxPortNumber and MinPortNumber. So the service is running, port number
may not be 8080 or what you specified. We put the selected port number in the log. 

3. This port number is tight to YARN RM. For end user that uses http service through Yarn,
they don't need to specify the port number on URL. 

Julia

-----Original Message-----
From: Markus Weimer [mailto:weimer@apache.org] 
Sent: Wednesday, October 08, 2014 9:06 AM
To: dev@reef.incubator.apache.org
Subject: Re: Question about Reef-webserver

Hi,

mostly: What Yingda said.

Typically, we are on clusters with shared machines where one physical machine (and IP address)
hosts multiple containers. Assuming that any one particular port is available for us isn't
possible. Hence, the constructor of HttpServerImpl tries different ports until an available
one is found. In the case of the Driver, we use the found port in order to register the application
with the resource manager to enable discovery through the RM, which is part of the infrastructure
of the cluster and therefore a known entity.


The way HttpServerImpl  does this searchfor a port  is configurable, but it starts with com.microsoft.reef.webserver.PortNumber,
which defaults to 8080. If you want to start the search elsewhere, you can bind a different
value to that named parameter.

Hope this helps,

Markus

On Wed, Oct 8, 2014 at 7:15 AM, Yingda Chen <ydchen@gmail.com> wrote:

> We had similar discussions before, and for most usage cases, we depend 
> on YARN to do the http traffic forwarding, which will automatically 
> find the port used by Reef (AM) http server.
>
> I agree that allowing user to set port can provide more flexibility, 
> but with flexibility also comes some uncertainty - since we cannot 
> guarantee that the port specified by the user can be used, as another 
> application may already be using that. In fact, on a multi-tenant 
> cluster where multiple Reef applications are running, port-collision 
> is inevitable. I guess we can still do it and make it clear to the 
> user that REEF can only bind the port on a best-effort basis. I am 
> open to that discussion :)
>
> -Yingda
>
> On Wed, Oct 8, 2014 at 3:21 AM, Yunseong Lee <yunseong.lee@me.com> wrote:
>
> > Hi, I am building an application using Reef-webserver and I have a 
> > question.
> >
> > Is there a way to set a port number other than the default value(8080)?
> >
> > As far as I have found, HttpServer does not have a method to set the 
> > port number.
> > I found some tests binding PortNumber via Configuration, but since 
> > PortNumber is not a public class it is not available outside the 
> > package.
> >
> > In my opinion, it is reasonable to allow user to set the port number.
> > When another application reserved 8080 port, I found the app and 
> > reef-webserver both go wrong.
> >
> > Any thoughts?
> >
> > Thank you.
> > - Yunseong
>
Mime
View raw message