hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James F Walton <jfwal...@us.ibm.com>
Subject Hadoop 1.0.3 (nutch-1.5.1) throwing errors on AIX 6.1
Date Fri, 17 Aug 2012 18:45:37 GMT
When implementing Nutch 1.0 awhile back, we had to point our scripts to 
/opt/freeware/bin to allow Nutch's Hadoop code to utilize the more 
Linux-like versions of various system commands (like df) in order for 
Hadoop to function properly on AIX.  After doing that, we've had no issues 
with Nutch's hadoop implementation from 1.0 through 1.4.

Now I'm attempting up migrate my existing, functional configurations from 
my Nutch 1.4 installation to Nutch 1.5.1, which is now using Hadoop 1.0.3.
Now when I attempt to run a crawl, I'm getting these errors that seem to 
be coming from Hadoop, and seem to want to use a LinuxLoginModule class 
(see below)

Is there some configuration setting, plugin or jar file, etc that is 
missing from the new version to make this all work on AIX again?

2012-08-02 20:03:21,271 ERROR crawl.Injector - Injector: 
java.lang.RuntimeException: java.io.IOException: failure to login
        at 
org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:546)
        at 
org.apache.hadoop.mapred.FileInputFormat.addInputPath(FileInputFormat.java:336)
        at org.apache.nutch.crawl.Injector.inject(Injector.java:209)
        at org.apache.nutch.crawl.Injector.run(Injector.java:248)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.nutch.crawl.Injector.main(Injector.java:238)
Caused by: java.io.IOException: failure to login
        at 
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:490)
        at 
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:452)
        at 
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1494)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1395)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)
        at 
org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:542)
        ... 5 more
Caused by: javax.security.auth.login.LoginException: unable to find 
LoginModule class: com.ibm.security.auth.module.LinuxLoginModule
        at 
javax.security.auth.login.LoginContext.invoke(LoginContext.java:834)
        at 
javax.security.auth.login.LoginContext.access$000(LoginContext.java:209)
        at 
javax.security.auth.login.LoginContext$5.run(LoginContext.java:732)
        at 
java.security.AccessController.doPrivileged(AccessController.java:284)
        at 
javax.security.auth.login.LoginContext.invokeCreatorPriv(LoginContext.java:729)
        at 
javax.security.auth.login.LoginContext.login(LoginContext.java:599)
        at 
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:471)
        ... 11 more



James
Mime
View raw message