hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Buckley <kevin.buckley.ecs.vuw.ac...@gmail.com>
Subject Re: Kerberised JobHistory Server not starting: User jhs trying to create the /mr-history/done directory
Date Mon, 24 Jul 2017 01:37:43 GMT
On 21 July 2017 at 13:25, Kevin Buckley
<kevin.buckley.ecs.vuw.ac.nz@gmail.com> wrote:
> On 21 July 2017 at 04:04, Erik Krogen <ekrogen@linkedin.com> wrote:
>> Hi Kevin,
>>
>> Since you are using the "jhs" keytab with principal "jhs/_HOST@REALM.TLD",
>> the JHS is authenticating itself as the jhs user (which is the actual
>> important part, rather than the user the process is running as). If you want
>> it to be the "mapred" user, you should change the keytab/principal you use
>> (mapred.jobhistory.{principal,keytab}).
>
> I'll certainly give that a go Erik, however, the way I read the
>
>>> The hadoop-2.8.0  docs SecureMode page also suggests that one would need to
>>> play around with the
>>>
>>> hadoop.security.auth_to_local
>
> bits suggested to me that if you set things up such that
>
> =======
> $ hadoop org.apache.hadoop.security.HadoopKerberosName
> jhs/co246a-9.ecs.vuw.ac.nz@ECS.VUW.AC.NZ
> 17/07/20 17:42:50 INFO util.KerberosName: Non-simple name
> mapred/co246a-9.ecs.vuw.ac.nz@ECS.VUW.AC.NZ after auth_to_local rule
> RULE:[2:$1/$2@$0](jhs/.*)s/jhs/mapred/
> Name: jhs/co246a-9.ecs.vuw.ac.nz@ECS.VUW.AC.NZ to
> mapred/co246a-9.ecs.vuw.ac.nz@ECS.VUW.AC.NZ
> ====
>
> (or even used a rule that just mapped the principal to a simple "mapred"
> because I tried that too !) told you it was remapping the user, then it would
> remap for all instances of the user, within the Hadoop instance..
>
> Let's see.

OK,

so it would appear that despite the Hadoop docs appearing to suggest that
you only need the three usernames, 'hdfs', 'yarn' and 'mapred'. if you do use
the principal from the docs, which has the jhs component, then even if you
do try to map users using 'hadoop.security.auth_to_local', your JobHistory
server will start up, inside Hadoop running as a 'jhs' user.

That would seem to be a bit of a trap for the unaware/unwary that the docs
could easily improve upon ?

Thanks again for the pointer to the correct interpreation of the docs, Erik,
Kevin

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org


Mime
View raw message