hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Leffel" <daniel.lef...@gmail.com>
Subject Why is there a seperate map and reduce task capacity?
Date Tue, 17 Jun 2008 04:19:05 GMT
Why not just combine them? How do I do that?

Rationale is that our tasks are very balanced in load, but unbalanced
in timing. I've found that limiting the number of total threads to be
the most safe approach to not overloading the dfs daemon. To date,
I've done that just through intelligent scheduling of jobs to stagger
maps and reduces, but have I missed a setting that exists to simply
limit number of tasks in-total?

View raw message