hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@yahoo-inc.com>
Subject Re: Ok to share ZK nodes with Hadoop nodes?
Date Mon, 08 Mar 2010 19:17:11 GMT
Hi David,
  Sharing the cluster with HDFS and Map reduce might cause significant
problems. Mapreduce is very IO intensive and this might cause lot of
unnecessary hiccups in your cluster. I would suggest atleast providing
something like this, if you really want to share the nodes.

- atleast considerable amount of memory space say 400-500MB (depending on
your usage) for the java heap
- one dedicated disk not used by MR or Datanodes, so that ZooKeeper
performance is a little predictable for you.

Thanks
mahadev


On 3/8/10 10:58 AM, "David Rosenstrauch" <darose@darose.net> wrote:

> I'm contemplating an upcoming zookeeper rollout and was wondering what
> the zookeeper brain trust here thought about a network deployment question:
> 
> Is it generally considered bad practice to just deploy zookeeper on our
> existing hdfs/MR nodes?  Or is it better to run zookeeper instances on
> their own dedicated nodes?
> 
> On the one hand, we're not going to be making heavy-duty use of
> zookeeper, so it might be sufficient for zookeeper nodes to share box
> resources with HDFS & MR.  On the other hand, though, I don't want
> zookeeper to become unavailable if the nodes are running a resource
> intensive job that's hogging CPU or network.
> 
> 
> What's generally considered best practice for Zookeeper?
> 
> Thanks,
> 
> DR


Mime
View raw message