hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khaled Elmeleegy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5632) Jobtracker leaves tasktrackers underutilized
Date Wed, 22 Apr 2009 15:21:47 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701570#action_12701570
] 

Khaled Elmeleegy commented on HADOOP-5632:
------------------------------------------



Coarse grained locking is only part of the problem. The second part and
likely a more important part is that the heartbeat handler is heavy weight.
The handler runtime was 3+ ms for my workload. For other workloads it can go
higher.

If we can bring down the handler's runtime to 500 us (which is well within
reach), and assume we have 4000 nodes cluster with 10 slots per node and
each map runs for 20 s. That would result in 2000 heartbeats per second. The
JT can keep up with that even with locking left unchanged. Obviously, we'll
need to fix the locking in the future, but I think that lighter weight
heartbeats can be a quicker fix.



> Jobtracker leaves tasktrackers underutilized
> --------------------------------------------
>
>                 Key: HADOOP-5632
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5632
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.18.0, 0.18.1, 0.18.2, 0.18.3, 0.19.0, 0.19.1, 0.20.0
>         Environment: 2x HT 2.8GHz Intel Xeon, 3GB RAM, 4x 250GB HD linux boxes, 100 node
cluster
>            Reporter: Khaled Elmeleegy
>         Attachments: hadoop-khaled-tasktracker.10s.uncompress.timeline.pdf, hadoop-khaled-tasktracker.150ms.uncompress.timeline.pdf,
jobtracker.patch, jobtracker20.patch
>
>
> For some workloads, the jobtracker doesn't keep all the slots utilized even under heavy
load.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message