hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavan Kulkarni <pavan.babu...@gmail.com>
Subject Re: Doubt regarding use of HTTP during Shuffle phase
Date Wed, 27 Jun 2012 16:57:47 GMT
@Roman: Exactly these are the papers I referred before venturing into this
project.
Thanks a lot for the sailfish paper.Wasn't aware of that.
Also by any chance do you have any idea what classes do I need to tweak to
get the Filenames of Map Output files.?Thanks

On Wed, Jun 27, 2012 at 9:52 AM, Roman Shaposhnik <rvs@apache.org> wrote:

> On Wed, Jun 27, 2012 at 9:44 AM, Pavan Kulkarni <pavan.baburao@gmail.com>
> wrote:
> > Yes you are correct, but we can use Lustre FS and it does scale right?
> > I am new to this so please excuse if I am wrong in some assumptions.
>
> You can use hybrid approaches, but you'd be venturing into the unknown.
> Take a look at these projects first:
>    http://code.google.com/p/sailfish/
>
> http://www.xyratex.com/pdfs/whitepapers/Xyratex_white_paper_MapReduce_1-4.pdf
>
> Thanks,
> Roman.
>



-- 

--With Regards
Pavan Kulkarni

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message