hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rishi Yadav" <ri...@infoobjects.com>
Subject Re: Gathering connection information
Date Sat, 07 Jun 2014 14:19:37 GMT
Typically users ssh edge node which is co-located with the cluster. It also minimizes latency
between client and cluster.




—
Sent from Mailbox

On Sat, Jun 7, 2014 at 7:12 AM, Peyman Mohajerian <mohajeri@gmail.com>
wrote:

> In my experience you build a node called Edge Node which has all the
> libraries and configuration setting in XML to connect to the cluster, it
> just doesn't have any of the Hadoop daemons running.
> On Wed, Jun 4, 2014 at 2:46 PM, John Lilley <john.lilley@redpoint.net>
> wrote:
>>  We’ve found that much of the Hadoop samples assume that running is being
>> done form a cluster node, and that the connection information can be
>> gleaned directly from a configuration object.  However, we always run our
>> client from a remote computer, and our users must manually specify the
>> NN/RM addresses and ports.  We’ve found this varies maddeningly between
>> distros and especially on hosted virtual implementations.  Getting the
>> wrong port results in various inscrutable errors with red-herring messages
>> about security.  Is there a prescribed way to get the correct connection
>> information more easily, like from a web API (where at least we’d only need
>> one address and port)?
>>
>>
>>
>> john
>>
>>
>>
Mime
View raw message