hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Collins <...@cloudera.com>
Subject Re: HDFS read/write speeds, and read optimization
Date Sun, 03 Jan 2010 00:02:40 GMT
Hey Stas,

Can you provide more information about your workload and the
environment? eg are you running t.o.a.h.h.BenchmarkThroughput,
TestDFSIO, or timing hadoop fs -put/get to transfer data to hdfs from
another machine, looking at metrics, etc. What else is running on the
cluster? Have you profiled? etc. 77Mb/s (<10MB/s) seems low but w/o
context is not meaningful.

Thanks,
Eli

On Sat, Jan 2, 2010 at 2:54 PM, Stas Oskin <stas.oskin@gmail.com> wrote:
> Hi.
>
> Can anyone advice on the subject below?
>
> Thanks!
>
> On Mon, Dec 28, 2009 at 9:01 PM, Stas Oskin <stas.oskin@gmail.com> wrote:
>
>> Hi.
>>
>> Going back to the subject, has anyone ever bench-marked small (10 - 20
>> node) HDFS clusters?
>>
>> I did my own speed checks, and it seems I can reach ~77Mbps, on a quad-disk
>> node. This comes to ~19Mbps per disk, which seems quite low in my opinion.
>>
>> Can anyone advice about this?
>>
>> Thanks.
>>
>

Mime
View raw message