Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
MIME-Version: 1.0
Date: Wed, 5 Aug 2015 23:02:46 +0530
Message-ID: 
 <CAGzZitU5k1GpdPfkeVH-2RLAgprJ7k9gUPRX9KZkWwJa7UYomQ@mail.gmail.com>
Subject: hadoop installation in pseudo distributed mode regular user vs
 dedicated user
From: Arvind Sundararajan <arvindappuser@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=94eb2c0763a682bb6d051c93ca69

--94eb2c0763a682bb6d051c93ca69
Content-Type: text/plain; charset=UTF-8

Hi All,

I have a laptop running Ubuntu 14.04 LTS and am trying to install hadoop
2.7.1 (current stable version) in pseudo-distributed mode.

I have a regular user account on my laptop, but am confused if i should
install hadoop using a dedicated hadoop user on my laptop.
NOTE: By 'regular user', i mean the linux user account that i use for
day-to-day personal work

The current hadoop documentation at [1] does not mention setting up a
dedicated user for hadoop installation.

However, the hadoop installation tutorial at [2] mentions setting up a
dedicated user for hadoop installation in pseudo-distributed mode on a
single machine. This tutorial references an outdated hadoop installation
tutorial [3] which too mentions setting up a dedicated user for hadoop
installation in pseudo-distributed mode on a single machine.

I found several tutorials online which all seem to mention setting up
dedicated user for hadoop installation in pseudo-distributed mode on a
single machine, without mentioning why we should set up a dedicated user.

My questions are as follows:

a) Is it possible for me to execute hadoop programs as a regular user even
if hadoop is installed in pseudo-distributed mode via a dedicated 'hadoop'
user?
If yes, what linux filesystem folder permissions and HDFS permissions do i
need to give to the regular user for executing hadoop programs?

b) Quoting from the outdated hadoop installation tutorial [3]:

    "We will use a dedicated Hadoop user account for running Hadoop.
     While that's not required it is recommended because it helps to separate
     the Hadoop installation from other software applications and
     user accounts running on the same machine
     (think: security, permissions, backups, etc)."

Can someone elaborate on this? what are the issues regarding security,
permissions, backups when running hadoop in pseudo-distributed mode on a
single laptop which will most likely have only one user account (my current
user account) ?

c) Can someone please elaborate on the pros and cons of running hadoop in
pseudo-distributed mode on a single machine as the regular user versus
creating a dedicated user?

My thoughts on the cons, thus far has been:

    i) if hadoop is unable to execute from a 'regular user' and
    only works from the dedicated hadoop user account, then i
    will have to edit my hadoop java programs from my
    'regular user' account where i have my development environment
    and IDE/text editor setup, copy the .jar files to the
    dedicated hadoop user account and execute. if any error occurs,
    i have to go back to the 'regular user' account, edit and
    then copy the new .jar files and execute again. this moving
    back and forth between accounts is a definite pain while
    working in pseudo-distributed mode and i have experienced
    this while working in Hadoop 1.x version

    ii) if hadoop is unable to execute from a 'regular user' and
    only works from the dedicated hadoop user account, then
    the hadoop operations copyFromLocal and copyToLocal will
    require a shared folder for both user accounts.

P.S. I also referred [4] and [5] before asking this question.

References:

[1]
http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/SingleCluster.html
[2] http://dogdogfish.com/big-data/installing-hadoop-2-4-on-ubuntu-14-04/
[3]
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
[4]
http://stackoverflow.com/questions/20192140/hadoop-pseudo-distributed-mode-for-multiple-users
[5]
http://stackoverflow.com/questions/23807486/hadoop-development-dedicated-user-in-ubuntu-how-to-access-hadoop-node-running

--94eb2c0763a682bb6d051c93ca69
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"post-text" itemprop=3D"text">

<p>Hi All,</p><p>I have a laptop running Ubuntu 14.04 LTS and am trying to =
install=20
hadoop 2.7.1 (current stable version) in pseudo-distributed mode.</p>

<p>I have a regular user account on my laptop, but am confused if i=20
should install hadoop using a dedicated hadoop user on my laptop.<br>
NOTE: By &#39;regular user&#39;, i mean the linux user account that i use f=
or day-to-day personal work</p>

<p>The current hadoop documentation at [1] does not mention setting up a de=
dicated user for hadoop installation.</p>

<p>However, the hadoop installation tutorial at [2] mentions setting up a
 dedicated user for hadoop installation in pseudo-distributed mode on a=20
single machine. This tutorial references an outdated hadoop installation
 tutorial [3] which too mentions setting up a dedicated user for hadoop=20
installation in pseudo-distributed mode on a single machine. </p>

<p>I found several tutorials online which all seem to mention setting up
 dedicated user for hadoop installation in pseudo-distributed mode on a=20
single machine, without mentioning why we should set up a dedicated=20
user.</p>

<p>My questions are as follows:</p>

<p>a)=20
    Is it possible for me to execute hadoop programs as a regular user=20
even if hadoop is installed in pseudo-distributed mode via a dedicated=20
&#39;hadoop&#39; user?<br>
If yes, what linux filesystem folder permissions and HDFS permissions do
 i need to give to the regular user for executing hadoop programs?</p>

<p>b)=20
    Quoting from the outdated hadoop installation tutorial [3]: </p>

<pre><code>    &quot;We will use a dedicated Hadoop user account for runnin=
g Hadoop.
     While that&#39;s not required it is recommended because it helps to se=
parate=20
     the Hadoop installation from other software applications and
     user accounts running on the same machine=20
     (think: security, permissions, backups, etc).&quot;
</code></pre>

<p>Can someone elaborate on this? what are the issues regarding=20
security, permissions, backups when running hadoop in pseudo-distributed
 mode on a single laptop which will most likely have only one user=20
account (my current user account) ?</p>

<p>c)=20
    Can someone please elaborate on the pros and cons of running hadoop=20
in pseudo-distributed mode on a single machine as the regular user=20
versus creating a dedicated user?</p>

<p>My thoughts on the cons, thus far has been:</p><pre><code>    i) if hado=
op is unable to execute from a &#39;regular user&#39; and=20
    only works from the dedicated hadoop user account, then i=20
    will have to edit my hadoop java programs from my=20
    &#39;regular user&#39; account where i have my development environment
    and IDE/text editor setup, copy the .jar files to the=20
    dedicated hadoop user account and execute. if any error occurs,=20
    i have to go back to the &#39;regular user&#39; account, edit and=20
    then copy the new .jar files and execute again. this moving=20
    back and forth between accounts is a definite pain while
    working in pseudo-distributed mode and i have experienced=20
    this while working in Hadoop 1.x version

    ii) if hadoop is unable to execute from a &#39;regular user&#39; and=20
    only works from the dedicated hadoop user account, then=20
    the hadoop operations copyFromLocal and copyToLocal will=20
    require a shared folder for both user accounts.
</code></pre>

<p>P.S. I also referred [4] and [5] before asking this question.</p>

<p>References:</p>

<p>[1] <a href=3D"http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/=
hadoop-common/SingleCluster.html">http://hadoop.apache.org/docs/r2.7.1/hado=
op-project-dist/hadoop-common/SingleCluster.html</a><br>
[2] <a href=3D"http://dogdogfish.com/big-data/installing-hadoop-2-4-on-ubun=
tu-14-04/" rel=3D"nofollow">http://dogdogfish.com/big-data/installing-hadoo=
p-2-4-on-ubuntu-14-04/</a><br>
[3] <a href=3D"http://www.michael-noll.com/tutorials/running-hadoop-on-ubun=
tu-linux-single-node-cluster/" rel=3D"nofollow">http://www.michael-noll.com=
/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/</a><br>
[4] <a href=3D"http://stackoverflow.com/questions/20192140/hadoop-pseudo-di=
stributed-mode-for-multiple-users">http://stackoverflow.com/questions/20192=
140/hadoop-pseudo-distributed-mode-for-multiple-users</a><br>
[5] <a href=3D"http://stackoverflow.com/questions/23807486/hadoop-developme=
nt-dedicated-user-in-ubuntu-how-to-access-hadoop-node-running">http://stack=
overflow.com/questions/23807486/hadoop-development-dedicated-user-in-ubuntu=
-how-to-access-hadoop-node-running</a></p></div></div>

--94eb2c0763a682bb6d051c93ca69--