hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abhay ratnaparkhi <abhay.ratnapar...@gmail.com>
Subject Re: NoSuchMethodError while calling a DAO method from Reducer
Date Mon, 09 May 2011 09:40:26 GMT
I found that XML files are getting cached somewhere while running MR task.
Even if my recently submitted job has new XML files (for ibatis queries),
it's using XML files from previously submitted job.

On Mon, May 9, 2011 at 1:27 PM, abhay ratnaparkhi <
abhay.ratnaparkhi@gmail.com> wrote:

>
> I checked the class file and it's having the new methods.
> It seems that previous class file for DAO is getting cached somewhere.
> Does hadoop caches some files in a job?
>
> Abhay
>
>
>
> On Tue, May 3, 2011 at 7:22 PM, abhay ratnaparkhi <
> abhay.ratnaparkhi@gmail.com> wrote:
>
>> I'm trying to run MR task to parse log files.
>> I'm trying to store parsed results in to DB2 tables.
>>
>> I'm making database connection from Reduce function and storing data to
>> database from there.
>> (not using *DBOutputFormat *as I don't know how to store data in multiple
>> tables by using that)
>>
>> I'm getting "*java.lang.NoSuchMethodError*" even though that method is
>> present and class has been compiled properly.
>> When I use same method in non MR code, it works fine.
>>
>> I've seen the class file from jar and the method is present there.
>> Can anyone suggest why does it's unable to call method even if it's there?
>>
>>
>> -05-03 09:35:00,575 FATAL org.apache.hadoop.mapred.TaskTracker: Error
>> running child : java.lang.NoSuchMethodError:
>> com/ibm/anonymization/data/AnalyticsDAO.insertUsage(Lcom/ibm/anonymization/beans/LogUsageRecord;)V
>>     at
>> com.ibm.anonymization.parser.MRLogParser$ParsingReducer.reduce(MRLogParser.java:411)
>>     at
>> com.ibm.anonymization.parser.MRLogParser$ParsingReducer.reduce(MRLogParser.java:1)
>>     at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
>>     at
>> org.apache.hadoop.mapred.Task$NewCombinerRunner.combine(Task.java:1222)
>>     at
>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1265)
>>     at
>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1129)
>>     at
>> org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:549)
>>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:623)
>>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>>     at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message