hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jdd dhok <jdd...@gmail.com>
Subject Re: Measuring IO time in map/reduce jobs?
Date Fri, 13 Feb 2009 09:10:49 GMT
Linux kernel provides delay accounting information through a netlink
socket to user space. You can read more about it here:
I think there's a python tool called iotop that uses this feature.

Hope this helps.


On Fri, Feb 13, 2009 at 2:40 AM, Bryan Duxbury <bryan@rapleaf.com> wrote:
> Hey all,
> Does anyone have any experience trying to measure IO time spent in their
> map/reduce jobs? I know how to profile a sample of map and reduce tasks, but
> that appears to exclude IO time. Just subtracting the total cpu time from
> the total run time of a task seems like too coarse an approach.
> -Bryan


View raw message