hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sampath Herga <sampat...@gmail.com>
Subject Re: Issue with third party library
Date Fri, 07 Dec 2012 07:25:34 GMT
Hi Hemanth,

Setting the full path worked.

Thanks,
Sampath.



On Thu, Dec 6, 2012 at 9:51 AM, Hemanth Yamijala
<yhemanth@thoughtworks.com>wrote:

> Sampath,
>
> You  mentioned that the file is present in the tasktracker local dir,
> could you please tell us the full path ? I am wondering if setting the full
> path will have any impact, rather than specifying the relative path.
>
> Another option may be to try to use the addCacheArchive and createSymLink
> APIs as described here:
> http://hadoop.apache.org/docs/stable/mapred_tutorial.html#DistributedCache
>
>
> On Wed, Dec 5, 2012 at 5:23 PM, Sampath Herga <sampath06@gmail.com> wrote:
>
>> Hi,
>>
>> I was just trying to setup hadoop and run some sample programs. I ran
>> into some issues when trying to add a required library. I tried using the
>> DistributedCache methods to try adding to the classpath.
>>
>> The basic code is:
>>   Path mysqlJar = new
>> Path("target/classes/META-INF/mysql-connector-java-3.1.12-bin.jar");
>>   fs.copyFromLocalFile(mysqlJar, mysqlJar);
>>   DistributedCache.addArchiveToClassPath(mysqlJar, jobConf, fs);
>>
>> I did a copyFromLocal since I wasnt sure where it would get picked from.
>> I tried both addArchiveToClassPath and addFileToClassPath. But with both, I
>> get a ClassNotFoundException.
>>
>> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> com.mysql.jdbc.Driver
>>     at
>> org.apache.hadoop.mapred.lib.db.DBInputFormat.configure(DBInputFormat.java:271)
>>     ... 16 more
>>
>> The job seems to have got the file added and also checked in the
>> tasktracker local dir and the file seems to be present.
>>
>>
>> <property><name>mapred.job.classpath.archives</name><value>target/classes/META-INF/mysql-connector-java-3.1.12-bin.jar</value></property>
>>
>> Any clues would be helpful. I dont think I can use -libjars since I am
>> trying to execute the hadoop jobs from within another application and not
>> from the command line.
>>
>> Regards,
>> Sampath.
>>
>
>

Mime
View raw message