hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Esteban Gutierrez Moguel <esteban...@gmail.com>
Subject Re: the performance of HDFS
Date Wed, 26 Jan 2011 17:14:29 GMT
Da,

Do you have permissions to run iostat? That could help you to find
persistent IO activity from other processes.

Another possibility is that your system has a RAID controller without a
cache memory card, from my experience that makes writes awfully slow.


esteban.


On Tue, Jan 25, 2011 at 20:29, Da Zheng <zhengda1936@gmail.com> wrote:

> No, each node in the cluster is powerful server. I was told the nodes are
> Dell
> Poweredge SC1435, but I cannot figure out the configuration of hard drives.
> Dell
> provides several possible hard drives for this model.
>
> On 1/25/11 7:59 PM, Ted Dunning wrote:
> > This is a really slow drive or controller.
> >
> > Consumer grade 3.5 inch 2TB drives typically can handle 100MB/s.  I would
> > suspect in the absence of real information that your controller is more
> > likely to be deficient than your drive.  If this is on a laptop or
> > something, then I withdraw my thought.
> >
> > On Tue, Jan 25, 2011 at 4:50 PM, Da Zheng <zhengda1936@gmail.com> wrote:
> >
> >> The aggregate write-rate can get much higher if you use more drives, but
> a
> >>> single stream throughput is limited to the speed of one disk spindle.
> >>>
> >>>  You are right. I measure the performance of the hard drive. It seems
> the
> >> bottleneck is the hard drive, but the hard drive is a little too slow.
> The
> >> average writing rate is 50MB/s.
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message