Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9F2EF10EC7 for ; Tue, 27 Jan 2015 21:21:56 +0000 (UTC) Received: (qmail 1732 invoked by uid 500); 27 Jan 2015 21:21:52 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 1614 invoked by uid 500); 27 Jan 2015 21:21:52 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 1603 invoked by uid 99); 27 Jan 2015 21:21:52 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Jan 2015 21:21:52 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [196.218.210.163] (HELO smtp.aossama.net) (196.218.210.163) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Jan 2015 21:21:25 +0000 Received: from [192.168.1.148] (unknown [41.38.22.240]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: ahmed) by smtp.aossama.net (Postfix) with ESMTPSA id 790953EA1 for ; Tue, 27 Jan 2015 23:20:37 +0200 (GMT-2) Message-ID: <54C80121.9050901@aossama.com> Date: Tue, 27 Jan 2015 23:20:33 +0200 From: Ahmed Ossama User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: user@hadoop.apache.org Subject: Re: Hadoop 2.6.0 Multi Node Setup References: <54C7DFD9.6020402@aossama.com> In-Reply-To: Content-Type: multipart/alternative; boundary="------------060909060806090501080601" X-Virus-Checked: Checked by ClamAV on apache.org This is a multi-part message in MIME format. --------------060909060806090501080601 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Make sure that all nodes can resolve each other. You can do this by simply modifying /etc/hosts on each node with the IPs of the cluster Then add them to your /etc/hadoop/slaves file. On 01/27/2015 10:58 PM, Telles Nobrega wrote: > I was able to start some services, but Yarn is failing with > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: > java.io.IOException: Failed on local exception: > java.net.SocketException: Unresolved address; Host Details : local > host is: "telles-hadoop-two"; destination host is: (unknown):0. > > Just to give an overview of my setup. I have 6 machines, they can talk > to each other passwordless. One is the master with NameNode and a > second master will run ResourceManager. The slaves will run > NodeManager and DataNode. > > NameNode and DataNodes are ok. ResourceManager is still failing. > > On Tue Jan 27 2015 at 16:49:24 Telles Nobrega > wrote: > > Thanks. > > On Tue Jan 27 2015 at 15:59:35 Ahmed Ossama > wrote: > > Hi Telles, > > No, the documentation isn't out of date. Normally hadoop > configuration files are placed under /etc/hadoop/conf, it then > referenced to when starting the cluster with --config > $HADOOP_CONF_DIR, this is how hdfs and yarn know their > configuration. > > Second, it's not a good practice to run hadoop with root. What > you want to do is something like this > > # useradd hdfs > # useradd yarn > # groupadd hadoop > # usermod -a -Ghadoop hdfs > # usermod -a -Ghadoop yarn > # mkdir /hdfs/{nn,dn} > # chown -R hdfs:hadoop /hdfs > > Then start your hdfs daemon with hdfs user, and yarn daemon > with yarn user. > > > On 01/27/2015 08:40 PM, Telles Nobrega wrote: >> Hi, I'm starting to deply Hadoop 2.6.0 multi node. >> My first question is: >> In the documenation page, it says that the configuration >> files are under conf/ but I found them in etc/. Should I move >> them to conf or is this just out of date information? >> >> My second question is regarding users permission, I tried >> installing before but I was only able to start deamons >> running as root, is that how it should be? >> >> For now these are all the question I have. >> >> Thanks > > -- > Regards, > Ahmed Ossama > -- Regards, Ahmed Ossama --------------060909060806090501080601 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit Make sure that all nodes can resolve each other.

You can do this by simply modifying /etc/hosts on each node with the IPs of the cluster

Then add them to your /etc/hadoop/slaves file.

On 01/27/2015 10:58 PM, Telles Nobrega wrote:
I was able to start some services, but Yarn is failing with 
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.IOException: Failed on local exception: java.net.SocketException: Unresolved address; Host Details : local host is: "telles-hadoop-two"; destination host is: (unknown):0.

Just to give an overview of my setup. I have 6 machines, they can talk to each other passwordless. One is the master with NameNode and a second master will run ResourceManager. The slaves will run NodeManager and DataNode. 

NameNode and DataNodes are ok. ResourceManager is still failing.

On Tue Jan 27 2015 at 16:49:24 Telles Nobrega <tellesnobrega@gmail.com> wrote:
Thanks. 

On Tue Jan 27 2015 at 15:59:35 Ahmed Ossama <ahmed@aossama.com> wrote:
Hi Telles,

No, the documentation isn't out of date. Normally hadoop configuration files are placed under /etc/hadoop/conf, it then referenced to when starting the cluster with --config $HADOOP_CONF_DIR, this is how hdfs and yarn know their configuration.

Second, it's not a good practice to run hadoop with root. What you want to do is something like this
# useradd hdfs
# useradd yarn
# groupadd hadoop
# usermod -a -Ghadoop hdfs
# usermod -a -Ghadoop yarn
# mkdir /hdfs/{nn,dn}
# chown -R hdfs:hadoop /hdfs
Then start your hdfs daemon with hdfs user, and yarn daemon with yarn user.


On 01/27/2015 08:40 PM, Telles Nobrega wrote:
Hi, I'm starting to deply Hadoop 2.6.0 multi node. 
My first question is:
In the documenation page, it says that the configuration files are under conf/ but I found them in etc/. Should I move them to conf or is this just out of date information?

My second question is regarding users permission, I tried installing before but I was only able to start deamons running as root, is that how it should be?

For now these are all the question I have.

Thanks

-- 
Regards,
Ahmed Ossama

-- 
Regards,
Ahmed Ossama
--------------060909060806090501080601--