hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13673) Update scripts to be smarter when running with privilege
Date Tue, 17 Jan 2017 21:59:26 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826906#comment-15826906
] 

Andrew Wang commented on HADOOP-13673:
--------------------------------------

Just for completeness, here's the email exchange between myself and Allen (I hope Allen doesn't
mind me posting this):

> * hadoop_abs, does {{readlink -f}} accomplish the same thing?

        Effectively yes, but unfortunately, readlink isn't POSIX. It works differently on
different operating systems, even to the point of having radically different parameters. So
we can't rely upon it.  hadoop_abs, while obviously slower, is super portable. :)

> * Was it intentional to remove hadoop_usage from start-secure-dns.sh? The stop script
still has a usage.

        I was going to replace it but I guess I got distracted.  haha. I'll put it back for
now.

> * Few typos seen while reviewing: "legimately" "optinally" "definied" "description"

        I think i got all of these.

> * I think there's an extra "resourcemanager" in this line:
>
> {code}
> +  hadoop_uservar_su yarn resourcemanager proxyserver "${HADOOP_YARN_HOME}/bin/yarn"
\
> {code}

        Yup, definitely.

> * IIUC we we call {{hadoop_uservar_su}} directly in {{start-dfs.sh}} which requires that
the user vars to be set when running as root. Noticed though that {{start-balancer.sh}} doesn't
do this. Is this intentional or an omission?

        Intentional.  All of the single daemon scripts will switch when they call the main
hdfs/mapred/... script.  For the others, --workers needs to get called with the appropriate
user so that we don't try to use root's ssh key unless we really were meant to (e.g., secure
datanode).

> * Wondering if more needs to be said in the docs about what commands support this. For
instance, HTTPFS is off on the side, but I guess that'll be fixed once John finishes the conversion
from Tomcat to Jetty. Are there any other gaps you're aware of?

        Of the daemons, yeah, httpfs is a big outlier.  The other ones are rumen and sls.
 Now that we have dynamic commands, we should probably make them inline as well.

> Update scripts to be smarter when running with privilege
> --------------------------------------------------------
>
>                 Key: HADOOP-13673
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13673
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: scripts
>    Affects Versions: 3.0.0-alpha1, 3.0.0-alpha2
>            Reporter: Allen Wittenauer
>            Assignee: Allen Wittenauer
>              Labels: security
>         Attachments: HADOOP-13673.00.patch, HADOOP-13673.01.patch, HADOOP-13673.02.patch,
HADOOP-13673.03.patch, HADOOP-13673.04.patch
>
>
> As work continues on HADOOP-13397, it's become evident that we need better hooks to start
daemons as specifically configured users.  Via the (command)_(subcommand)_USER environment
variables in 3.x, we actually have a standardized way to do that.  This in turn means we can
make the sbin scripts super functional with a bit of updating:
> * Consolidate start-dfs.sh and start-secure-dns.sh into one script
> * Make start-\*.sh and stop-\*.sh know how to switch users when run as root
> * Undeprecate start/stop-all.sh so that it could be used as root for production purposes
and as a single user for non-production users



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message