hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavan Sudheendra <pavan0...@gmail.com>
Subject Re: How is pig so much faster than my java MR job?
Date Tue, 03 Sep 2013 06:03:32 GMT
Hi all,
I'm doing a kind of table join across 3 tables in the MR job ( plus doing
some computation).. It took nearly 19 hours to run with 21 mappers and 21
reducers.. But with pig it ran in less than 2 hours..
We are using HBase both as source and sink.. Is this normal?


On Mon, Sep 2, 2013 at 7:20 PM, Adrien Mogenet <adrien.mogenet@gmail.com>wrote:

> You should have a kind of debug/explain mode in Pig, and will show you how
> it does clever things to optimize its excution path.
>
>
> On Mon, Sep 2, 2013 at 3:45 PM, Dhaval Shah <prince_mithibai@yahoo.co.in
> >wrote:
>
> > Java MR code is not optimized/efficiently written while Pig is highly
> > optimized? Can you give us more details on what exactly you are trying to
> > do and how your Java MR code is written, how many MR jobs for Java vs Pig
> > and so on
> >
> > Sent from Yahoo! Mail on Android
> >
> >
>
>
> --
> Adrien Mogenet
> http://www.borntosegfault.com
>



-- 
Regards-
Pavan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message