hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brice Arnould (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3524) JobTracker's processHeartbeat() should not call System.currentTimeMillis() everytime
Date Tue, 10 Jun 2008 09:59:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12603822#action_12603822

Brice Arnould commented on HADOOP-3524:

Please excuse me, but I really don't think that it is a good idea.
The overall cost of having another thread to maintain would probably be greater than just
calling that function. In fact, even the access to the a field might take more time than calling
System.currentTimeMillis() (please refer to the attached test). And even if it divided the
cost of getting the current time by 100, that would probably have a negligible impact on performance,
with the cost of a greater complexity.
So I agree that there is many things to improve in the JobTracker, but not that.

(Please excuse me if I've been disrespectfull by inadvertance, I have difficulties with English

> JobTracker's processHeartbeat() should not call System.currentTimeMillis() everytime
> ------------------------------------------------------------------------------------
>                 Key: HADOOP-3524
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3524
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amar Kamat
> Consider the following
> {code:title=JobTracker.java|borderStyle=solid}
> private synchronized boolean processHeartbeat(
>                                                 TaskTrackerStatus trackerStatus, boolean
initialContact) {
>     String trackerName = trackerStatus.getTrackerName();
>     trackerStatus.setLastSeen(System.currentTimeMillis());
> {code}
> Here, the call to {{System.currentTimeMillis()}} on every call to {{JobTracker.processHeartbeat()}}
might prove costly. While testing/benchmarking HADOOP-2119, we recorded that the JobTracker
was able to serve ~130 tasks/sec. So that means we might make ~130 calls to {{System.currentTimeMillis()}}
per second. I think in these cases (_last-seen-status_ etc) such a high level of accuracy
in terms of timestamp is unnecessary and hence can be avoided.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message