hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Lilley <john.lil...@redpoint.net>
Subject RE: winutils and security
Date Sun, 24 Aug 2014 17:04:45 GMT
Following up on this, I was able to extract a winutils.exe and Hadoop.dll from a Hadoop install
for Windows, and set up HADDOP_HOME and PATH to find them.  It makes no difference to security,


From: John Lilley [mailto:john.lilley@redpoint.net]
Sent: Saturday, August 23, 2014 2:41 PM
To: 'user@hadoop.apache.org'
Subject: winutils and security

Up to this point, we've been able to run as a Hadoop client application (HDFS + YARN) from
Windows without winutils.exe, despite always seeing messages complaining about it in the logs.
 However, we are now integrating with secure clusters and are having some mysterious errors.
 Before these errors occur, messages come from Hadoop like those below.  Is it possible that
this is leading to our security failures? (I posted previously about that problem but got
no response).  What does winutils.exe have to do with security, if anything?


The relevant portions of the log seem to be:

2014-08-23 14:33:10 DEBUG org.apache.hadoop.metrics2.impl.MetricsSystemImpl: UgiMetrics, User
and group related metrics
2014-08-23 14:33:10 DEBUG org.apache.hadoop.security.Groups:  Creating new Groups object
2014-08-23 14:33:10 DEBUG org.apache.hadoop.util.NativeCodeLoader: Trying to load the custom-built
native-hadoop library...
2014-08-23 14:33:10 DEBUG org.apache.hadoop.util.NativeCodeLoader: Failed to load native-hadoop
with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
2014-08-23 14:33:10 DEBUG org.apache.hadoop.util.NativeCodeLoader: java.library.path=[...]
2014-08-23 14:33:10 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using
builtin-java classes where applicable
2014-08-23 14:33:10 DEBUG org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback:
Falling back to shell based
2014-08-23 14:33:10 DEBUG org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback:
Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
2014-08-23 14:33:10 DEBUG org.apache.hadoop.security.Groups: Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback;

View raw message