hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vjeran Marcinko" <vjeran.marci...@email.t-com.hr>
Subject Collision with Hadoop (1.0.4) libs?
Date Fri, 19 Apr 2013 21:03:10 GMT
Hi,

 

I created "fat jar" to run my M/R "driver" application, and this fat jar
contains beside other libs:

slf4j-api-1.7.5.jar

slf4j-simple-1.7.5.jar

. and to delegate all commons-logging calls to slf4j.

jcl-over-slf4j-1.7.5.jar  

 

Unfortunatelly, when I start my application using jar command:

"hadoop jar myfat.jar ."

I get following:

 

13/04/19 22:50:21 INFO property.AppProps: using app.id:
F4278122BFBA5B98991997F8B15E68F6

Exception in thread "main" java.lang.NoSuchMethodError:
org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;IL
java/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V

            at
org.apache.commons.logging.impl.SLF4JLocationAwareLog.debug(SLF4JLocationAwa
reLog.java:133)

            at
org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:
139)

            at
org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformat
ion.java:205)

            at
org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupI
nformation.java:184)

            at
org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupI
nformation.java:236)

            at
org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInform
ation.java:466)

            at
org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInfo
rmation.java:452)

            at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1494)

            at
org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1395)

            at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254)

            at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)

            at cascading.tap.hadoop.Hfs.getDefaultFileSystem(Hfs.java:307)

 

It seems that my application uses my bundled "jcl-over-slf4j-1.7.5.jar" to
delegate calls to some older slf4j-api.jar (not my bundled 1.7.5 version of
SLF4J), and I guess it can only be "slf4j-api-1.4.3.jar" found under
<hadoop_home>/lib ? That means that this old hadoop's slf4j lib has
preedence over the one that came bundled within my fat jar?!

 

Any suggestion to resolve this?

 

Regards,

Vjeran

 


Mime
View raw message