hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajat Goel <rajatgoe...@gmail.com>
Subject Profiling HDFS
Date Wed, 29 Dec 2010 05:45:07 GMT

I want to measure read/write rates to HDFS under various conditions such as
under heavy load or one data node goes down etc? Is there some profiler
already available for such purpose?

I am pushing data at high rate to HDFS, reads are also happening in parallel
and I suddenly reboot one datanode. I observe that I am no longer able to
write to HDFS (from live datanodes) at the same higher rate. This happens
for few minutes (around 30 mins), after which things go back to normal
again. I want to find out why HDFS becomes slow, what is the main
contributor of this latency and can I improve this behavior by changing some
configuration parameters.

Thanks & Regards,

View raw message