hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dhruba Borthakur <dhr...@gmail.com>
Subject Re: Profiling HDFS
Date Wed, 29 Dec 2010 05:51:41 GMT
also, try dfstestio and then restart one datanode and see how it behaves


On Tue, Dec 28, 2010 at 9:45 PM, Rajat Goel <rajatgoel06@gmail.com> wrote:

> Hi,
> I want to measure read/write rates to HDFS under various conditions such as
> under heavy load or one data node goes down etc? Is there some profiler
> already available for such purpose?
> I am pushing data at high rate to HDFS, reads are also happening in
> parallel and I suddenly reboot one datanode. I observe that I am no longer
> able to write to HDFS (from live datanodes) at the same higher rate. This
> happens for few minutes (around 30 mins), after which things go back to
> normal again. I want to find out why HDFS becomes slow, what is the main
> contributor of this latency and can I improve this behavior by changing some
> configuration parameters.
> Thanks & Regards,
> Rajat

Connect to me at http://www.facebook.com/dhruba

View raw message