hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Payne <er...@yahoo-inc.com>
Subject HDFS on trunk is now quite slow
Date Fri, 01 Jul 2011 19:57:36 GMT
Hi all,

I ran some stress tests on the latest HDFS trunk yesterday, and the performance is a lot slower
(sometimes 10 times slower) when compared with the HDFS in MR-279. The HDFS in MR-279 is slightly
behind trunk. The stability of the namenode in trunk seems to be better than in MR-279 (specifically,
HDFS:1257), but I'm not sure if the slowness is avoiding the race contitions or if they are
actually fixed in trunk.

At this point, I'm not sure what is causing this performance disparity. I notice that Block
management has recently undergone significant changes in trunk. It has some new locking and
it is now in its own package.

Could that be part of the cause?


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message