hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Webster <michael.webs...@bronto.com>
Subject Re: Why do i get the following error all of a sudden?
Date Tue, 29 Oct 2013 19:31:16 GMT
I have seen this issue as well.  The problem seems to come down to an issue in 
TableMapReduceUtil#findOrCreateJars.  This class loads dependency jars into
the distributed cache for MR jobs, this class in turn calls JarFinder#getJar
from hadoop.  The getJar method  returns a jar file containing all of the
dependency classes. If it can find the dependency as jar, 
based on the protocol of the path URL, it just provides the path of the jar
file.  If it finds the class as a file it attempts to create a directory, by
default CWD/target/temp-dir, and build a jar there.  If the user submitting
the job doesn't have permissions to create that directory you get an
IOException.  Some possible solutions are:

1. chmod the directory you run the script from such that the user submitting
the job has permissions (i have verified that this works)
2. add a jar to the classpath containing the classes that you need, 
that should cause JarFinder#getJar to just provide the path of the jar file 
(I haven't yet tested this out yet, but it should work)

- michael

View raw message