hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiaobo Gu <guxiaobo1...@gmail.com>
Subject Re: FW: How to run multiple data nodes and multiple task trackers on single server.
Date Sun, 03 Jul 2011 14:07:38 GMT
Hi Harsh
I have successfully running 2 data nodes in a single vitual machine,
and we will depoly 4 or 8 data nodes on our big SMP server, which has
32 CPU cores and 256G RAM,in order to take full advantage of all the
resources, do we need to configure more task trackers too, or can we
set mapred.tasktracker.map.tasks.maximum and
mapred.tasktracker.reduce.tasks.maximum to a larger number such as 8
or 16 to achieve the same purpose?

We have seen this
http://developer.yahoo.com/events/hadoopsummit2011/agenda.html#21, but
have not get any more details, we think multiple data node
configuration on big SMP servers is a good point to start with.


Xiaobo Gu

On Sun, Jul 3, 2011 at 9:56 PM, Harsh J <harsh@cloudera.com> wrote:
> On Sun, Jul 3, 2011 at 9:41 AM, XiaoboGu <guxiaobo1982@gmail.com> wrote:
>>> Hi,
>>> Do we have to run multiple task trackers when running multiple data nodes on
a single
>>> computer?
>>> Regards,
>>> Xiaobo Gu
> Do we _have_ to? --> No, its a matter of your choice if you want
> MapReduce daemons running along. They are not coupled.
> Regd. your original question, what's the string of "$DN_CONF_OPTS" being passed?
> --
> Harsh J

View raw message