hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Datskos <george.dats...@jp.fujitsu.com>
Subject Re: How to get the HDFS I/O information
Date Wed, 25 Apr 2012 01:06:03 GMT

Every job has a history file that is, by default, stored under 
$HADOOP_LOG_DIR/history.  These "job history" files list the amount of 
hdfs read/write (and lots of other things) for every task.

On 2012/04/25 7:25, Qu Chen wrote:
> Let me add, I'd like to do this periodically to gather some 
> performance profile information.
> On Tue, Apr 24, 2012 at 5:47 PM, Qu Chen <chenqu123@gmail.com 
> <mailto:chenqu123@gmail.com>> wrote:
>     I am trying to gather the info regarding the amount of HDFS
>     read/write for each task in a given map-reduce job. How can I do that?

View raw message