hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: HDFS on trunk is now quite slow
Date Sat, 02 Jul 2011 00:48:42 GMT
My guess is HDFS-988 caused the slowdown by coarsening some locking that was
previously incorrect. Your stress test is NN-only (metadata ops), not an I/O
benchmark, right? I/O should be faster in trunk than ever before.

-Todd

On Fri, Jul 1, 2011 at 8:23 AM, Eric Payne <eric.payne1000@yahoo.com> wrote:

> Hi gang,
>
> I ran some stress tests on the latest HDFS trunk yesterday, and the
> performance
> is a lot slower (sometimes 10 times slower) when compared with the HDFS in
> MR-279. The HDFS in MR-279 is slightly behind trunk. The stability of
> HDFS trunk
> seems to be better than HDFS MR-279, but I'm not sure if the slowness is
> just
> avoiding the race contitions or if they are actually fixed in trunk.
>
> At this point, I'm not sure what is causing this performance disparity. I
> notice
> that Block management has recently undergone significant changes in trunk.
> It
> has some new locking and it is now in its own package. Could this be part
> of the
> cause?
>
> Thanks,
> -Eric




-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message