Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <820733.17823.qm@web65509.mail.ac4.yahoo.com>
Date: Mon, 5 Apr 2010 11:41:05 -0700 (PDT)
From: Andrew Purtell <apurtell@apache.org>
Reply-To: apurtell@apache.org
Subject: Re: About test/production server configuration
To: hbase-user@hadoop.apache.org
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

The below from Patrick is not uncommon to encounter.=0A=0AThe "commodity ha=
rdware" talk around MR and BigTable is a bit of a joke -- you can do that i=
f you can afford 1,000s or 10,000s of commodity components custom assembled=
. Hadoop+HBase users want to do more with less, obviously. Colocating compu=
tation with storage has its price -- either you horizontally scale wide or =
go vertical enough on each node to handle the load you are throwing at the =
cluster you can afford. =0A=0ASizing clusters is a black art. =0A=0AAs for =
the spec of each individual node, I can share our current generation hardwa=
re spec:=0A=0A=A0 CPU: dual 6-core AMD (12 cores total)=0A=A0 RAM: 32 GB=0A=
=A0 DISK: 320 GB x 2 (RAID-1) system disk=0A=A0 =A0 =A0 =A0 500 GB x 8 (JBO=
D) data disks for HDFS=0A=A0 custom 1U chassis=0A=0A=A0 We give 8 GB of RAM=
 to the HBase region servers. All other Hadoop and HBase daemons (DataNode,=
 ZooKeeper, TaskTracker, etc.) use the default of 1 GB. Remainder of CPU an=
d RAM is for user tasks (MR).=0A=0A=A0 Reads are best served from RAM via t=
he block cache.=0A=0A=A0 The more spindles, the higher I/O parallelism, the=
refore higher aggregate throughput.=0A=0A=A0 The above is a good trade off =
between horizontal and vertical for us.=0A=0AHope that helps.=0A=0A> From: =
Patrick Hunt=0A> Subject: Re: About test/production server configuration=0A=
> The ZK servers are sensitive to disk=0A> (io) latency. I just troubleshot=
 an=0A> issue last week where a user was seeing 80second (second!)=0A> late=
ncies. Turns out they were running zk server, namenode,=0A> tasktracker, an=
d hbase region server all on the same box, =0A> that box had a single spind=
le for all io activity and was=0A> at 100% utilization for long periods of =
time. If=0A> you want decent ZK API latencies (<100ms) you really=0A> need =
to ensure that there's at least a separate spindle=0A> available for the ZK=
 transaction logs.=0A=0A=0A=0A