hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evert Lammerts <Evert.Lamme...@sara.nl>
Subject RE: Stability issue - dead DN's
Date Fri, 13 May 2011 09:54:40 GMT
Hi Mike,

> You really really don't want to do this.
> Long story short... It won't work.

Can you elaborate? Are you talking about the bonded interfaces or about having a separated
network for interconnects and external network? What can go wrong there?

>
> Just a suggestion.. You don't want anyone on your cluster itself. They
> should interact wit edge nodes, which are 'Hadoop aware'. Then your
> cluster has a single network to worry about.

That's our current setup. We have a single headnode that is used as a SPOE. However, I'd like
to change that on our future production system. We want to implement Kerberos for authentication,
and let users interact with the cluster from their own machine. This would enable them to
submit their jobs from the local IDE. The only way to do this is by opening up Hadoop ports
for the world, is my understanding: if people interact with HDFS they need to be able to interact
with all nodes, right? What would be the argument against this?

Cheers,
Evert

>
>
> Sent from a remote device. Please excuse any typos...
>
> Mike Segel
>
> On May 11, 2011, at 11:45 AM, Allen Wittenauer <aw@apache.org> wrote:
>
> >
> >
> >
> >
> >>> * a 2x1GE bonded network interface for interconnects
> >>> * a 2x1GE bonded network interface for external access
> >
> >    Multiple NICs on a box can sometimes cause big performance
> problems with Hadoop.  So watch your traffic carefully.
> >
> >
> >

Mime
View raw message