avro-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vyacheslav Zholudev <vyacheslav.zholu...@gmail.com>
Subject Re: Map output records/reducer input records mismatch
Date Tue, 16 Aug 2011 22:56:19 GMT
Hi, Scott,

thanks for your reply.

> What Avro version is this happening with? What JVM version?

We are using Avro 1.5.1 and Sun JDK 6, but the exact version I will have to look up.

> 
> On a hunch, have you tried adding -XX:-UseLoopPredicate to the JVM args if
> it is Sun and JRE 6u21 or later? (some issues in loop predicates affect
> Java 6 too, just not as many as the recent news on Java7).
> 
> Otherwise, it may likely be the same thing as AVRO-782.  Any extra
> information related to that issue would be welcome.

I will have to collect it. In the meanwhile, do you have any reasonable explanations of the
issue besides it being something like AVRO-782?

Thanks a lot,
Vyacheslav

> 
> Thanks!
> 
> -Scott
> 
> 
> 
> On 8/16/11 8:39 AM, "Vyacheslav Zholudev" <vyacheslav.zholudev@gmail.com>
> wrote:
> 
>> Hi,
>> 
>> I'm having multiple hadoop jobs that use the avro mapred API.
>> Only in one of the jobs I have a visible mismatch between a number of map
>> output records and reducer input records.
>> 
>> Does anybody encountered such a behavior? Can anybody think of possible
>> explanations of this phenomenon?
>> 
>> Any pointers/thoughts are highly appreciated!
>> 
>> Best,
>> Vyacheslav
> 
> 

Best,
Vyacheslav




Mime
View raw message