From zookeeper-user-return-1386-apmail-hadoop-zookeeper-user-archive=hadoop.apache.org@hadoop.apache.org Tue Mar 09 04:39:16 2010 Return-Path: Delivered-To: apmail-hadoop-zookeeper-user-archive@minotaur.apache.org Received: (qmail 85304 invoked from network); 9 Mar 2010 04:39:16 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 9 Mar 2010 04:39:16 -0000 Received: (qmail 17077 invoked by uid 500); 9 Mar 2010 04:38:50 -0000 Delivered-To: apmail-hadoop-zookeeper-user-archive@hadoop.apache.org Received: (qmail 16941 invoked by uid 500); 9 Mar 2010 04:38:50 -0000 Mailing-List: contact zookeeper-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: zookeeper-user@hadoop.apache.org Delivered-To: mailing list zookeeper-user@hadoop.apache.org Received: (qmail 16933 invoked by uid 99); 9 Mar 2010 04:38:49 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Mar 2010 04:38:49 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ted.dunning@gmail.com designates 74.125.92.24 as permitted sender) Received: from [74.125.92.24] (HELO qw-out-2122.google.com) (74.125.92.24) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Mar 2010 04:38:41 +0000 Received: by qw-out-2122.google.com with SMTP id 5so1779188qwi.35 for ; Mon, 08 Mar 2010 20:38:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :from:date:message-id:subject:to:content-type; bh=M3e6rDIpdGJIRSQc3IyJ9KmZ7XWClLHdEoUdbKEjios=; b=VphE3hAUWKTOf3mAOuEWKg40CLNh0QbXPi1LumFeN7ILAzbtN9SDOYW2DOvQEGfQq9 xKG/dRI1L5J3PR8Z0iF0Mr8YVPYZfRGT2XTS11dVcrwOmGhvIFm0rR8Cijj/Gvtj1O/l S4zKXwfJFizACiQNQW2llUt+uiwFczTsYflek= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=AO2qSbHQxhnR2Yg+NrVAZ2poYo1E6iGCSY+5R6f+gOzpy+3PvHck+grfZGzDWGGkx7 s8Edv9EFmGr7Vt8SSN186xfWT+pZZmOZv4MFJhXgDMGKQLV6+aaWNZQusn2FIgPl00p4 BB86ogctpgVX8EV0adcR6uxILV7WhJ1a38lEQ= MIME-Version: 1.0 Received: by 10.220.48.143 with SMTP id r15mr3307712vcf.95.1268109500753; Mon, 08 Mar 2010 20:38:20 -0800 (PST) In-Reply-To: <4B954E28.5040407@apache.org> References: <4B9548ED.7050608@darose.net> <4B954E28.5040407@apache.org> From: Ted Dunning Date: Mon, 8 Mar 2010 20:37:59 -0800 Message-ID: Subject: Re: Ok to share ZK nodes with Hadoop nodes? To: zookeeper-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=0016e648d28054294d048156c0cd X-Virus-Checked: Checked by ClamAV on apache.org --0016e648d28054294d048156c0cd Content-Type: text/plain; charset=UTF-8 I have used 5 and 3 in different clusters. Moderate amounts of sharing is reasonable, but sharing with less intensive applications is definitely better. Sharing with the job tracker, for instance is likely fine since it doesn't abuse disk so much. The namenode is similar, but not quite as nice. Sharing with task only nodes is better than sharing with data nodes. If your hadoop cluster is 10 machines, this is probably pretty serious overhead. If it is 200 machines, it is much less so. If you are running in EC2, then spawning 3 extra small instances is not a big deal. For the record, we share our production ZK machines with other tasks, but not with map-reduce related tasks and not with our production search engines. On Mon, Mar 8, 2010 at 11:21 AM, Patrick Hunt wrote: > Best practice for "on-line production serving" is 5 dedicated hosts with > "shared nothing", physically distributed thoughout the data center (5 hosts > in a rack might not be the best idea for super reliability). There's alot of > lee-way though, many ppl run with 3 and spof on switch for example. > --0016e648d28054294d048156c0cd--