hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andreas Kostyrka <andr...@kostyrka.org>
Subject Re: HDFS read/write speeds, and read optimization
Date Sun, 03 Jan 2010 03:21:19 GMT
Well, that all depends on many details, but:

-) are you really using 4 discs (configured correctly as data
directories?)

-) What hdd/connection technology?

-) And 77MB/s would match up curiously well with 1Gbit networking cards?
So you sure that you are testing a completely local setup? Where's your
name node running then?

Andreas 


Am Sonntag, den 03.01.2010, 00:54 +0200 schrieb Stas Oskin:
> Hi.
> 
> Can anyone advice on the subject below?
> 
> Thanks!
> 
> On Mon, Dec 28, 2009 at 9:01 PM, Stas Oskin <stas.oskin@gmail.com> wrote:
> 
> > Hi.
> >
> > Going back to the subject, has anyone ever bench-marked small (10 - 20
> > node) HDFS clusters?
> >
> > I did my own speed checks, and it seems I can reach ~77Mbps, on a quad-disk
> > node. This comes to ~19Mbps per disk, which seems quite low in my opinion.
> >
> > Can anyone advice about this?
> >
> > Thanks.
> >


Mime
View raw message