hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <...@apache.org>
Subject Re: Doubt regarding use of HTTP during Shuffle phase
Date Wed, 27 Jun 2012 16:52:00 GMT
On Wed, Jun 27, 2012 at 9:44 AM, Pavan Kulkarni <pavan.baburao@gmail.com> wrote:
> Yes you are correct, but we can use Lustre FS and it does scale right?
> I am new to this so please excuse if I am wrong in some assumptions.

You can use hybrid approaches, but you'd be venturing into the unknown.
Take a look at these projects first:
    http://code.google.com/p/sailfish/
    http://www.xyratex.com/pdfs/whitepapers/Xyratex_white_paper_MapReduce_1-4.pdf

Thanks,
Roman.

Mime
View raw message