hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hari Sekhon (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11400) Automatic HDFS Home Directory Creation
Date Fri, 10 Feb 2017 16:05:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15861450#comment-15861450
] 

Hari Sekhon edited comment on HDFS-11400 at 2/10/17 4:05 PM:
-------------------------------------------------------------

[~aw]

bq. If I access a home dir as a privileged user (e.g., hdfs) then I'm not sure why there would
be a validation made against an individual user's external existence.

That's not the use case - it's only when an actual user tries to do something in hdfs and
there is no home directory detected for that same user - this does not apply to hdfs superuser
operations at all - in fact validating "against an external user's existence" when touching
a home directory is the check in the wrong direction entirely.

This is more for jobs run by a user for which a home dir wasn't set up (the users just pop
up and start using the cluster in large enterprises as they're in some other part of the enterprise
that you never see but are added in an AD group that is allowed on the cluster - they could
be new guys or just someone you just never met because it's a big company).

bq. Whoever is building this on a per client basis ...

Ever tried copying your pre-written code from your github or private machine to Banks, government
environments and large traditional enterprises where everything is firewalled off, the internet
is blocked to server networks and nothing is allowed in or out? Write it again :-/ . Most
people in those types of places just have a dumb sheet that they have to follow for every
single person who requests to use the cluster as their jobs fail otherwise... they're lucky
if somebody even scripts it for them.

Yes it's only a couple of commands but people in those types of environments don't know anything
- which may be hard to understand how bad it is if you're used to working for tech startups
with smart techies and little security - so you have to script it again for them to happen
behind the scenes.

bq. Also, doesn't the NN plugin system already give one a way to implement this feature without
clogging up the rest of the code base?

If such a plugin is bundled and available in core hdfs and enabled with a simple config change
then ok but otherwise that idea is Dead-on-Arrival in a large chunk of verticals which do
not allow downloading and installing random things from the internet, which includes pretty
much all banks in the world, government departments and large traditional enterprises.

FYI in large environments the account validation and group memberships are handled by people
you never see through internal request systems, Hadoop administrators never touch those things
beyond the initial setup of which groups are allowed on the cluster, from then onwards all
new users and group memberships etc are handled by Active Directory teams that you never see
because they're in some other part of the large organization, and possible in different geographic
locations.


was (Author: harisekhon):
bq. If I access a home dir as a privileged user (e.g., hdfs) then I'm not sure why there would
be a validation made against an individual user's external existence.

That's not the use case - it's only when an actual user tries to do something in hdfs and
there is no home directory detected for that same user - this does not apply to hdfs superuser
operations at all - in fact validating "against an external user's existence" when touching
a home directory is the check in the wrong direction entirely.

This is more for jobs run by a user for which a home dir wasn't set up (the users just pop
up and start using the cluster in large enterprises as they're in some other part of the enterprise
that you never see but are added in an AD group that is allowed on the cluster - they could
be new guys or just someone you just never met because it's a big company).

bq. Whoever is building this on a per client basis ...

Ever tried copying your pre-written code from your github or private machine to Banks, government
environments and large traditional enterprises where everything is firewalled off, the internet
is blocked to server networks and nothing is allowed in or out? Write it again :-/ . Most
people in those types of places just have a dumb sheet that they have to follow for every
single person who requests to use the cluster as their jobs fail otherwise... they're lucky
if somebody even scripts it for them.

Yes it's only a couple of commands but people in those types of environments don't know anything
- which may be hard to understand how bad it is if you're used to working for tech startups
with smart techies and little security - so you have to script it again for them to happen
behind the scenes.

bq. Also, doesn't the NN plugin system already give one a way to implement this feature without
clogging up the rest of the code base?

If such a plugin is bundled and available in core hdfs and enabled with a simple config change
then ok but otherwise that idea is Dead-on-Arrival in a large chunk of verticals which do
not allow downloading and installing random things from the internet, which includes pretty
much all banks in the world, government departments and large traditional enterprises.

FYI in large environments the account validation and group memberships are handled by people
you never see through internal request systems, Hadoop administrators never touch those things
beyond the initial setup of which groups are allowed on the cluster, from then onwards all
new users and group memberships etc are handled by Active Directory teams that you never see
because they're in some other part of the large organization, and possible in different geographic
locations.

> Automatic HDFS Home Directory Creation
> --------------------------------------
>
>                 Key: HDFS-11400
>                 URL: https://issues.apache.org/jira/browse/HDFS-11400
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: hdfs, namenode
>    Affects Versions: 2.7.1
>         Environment: HDP 2.4.2
>            Reporter: Hari Sekhon
>
> Feature Request to add automatic home directory creation for HDFS users when they are
first resolved by the NameNode if their home directory does not already exist, using configurable
umask defaulting to 027.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message