hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Payne <eric.payne1...@yahoo.com>
Subject HDFS on trunk is now quite slow
Date Fri, 01 Jul 2011 15:23:35 GMT
Hi gang,

I ran some stress tests on the latest HDFS trunk yesterday, and the performance 
is a lot slower (sometimes 10 times slower) when compared with the HDFS in 
MR-279. The HDFS in MR-279 is slightly behind trunk. The stability of HDFS trunk 
seems to be better than HDFS MR-279, but I'm not sure if the slowness is just 
avoiding the race contitions or if they are actually fixed in trunk.

At this point, I'm not sure what is causing this performance disparity. I notice 
that Block management has recently undergone significant changes in trunk. It 
has some new locking and it is now in its own package. Could this be part of the 

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message