Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of yaojingguo@gmail.com
 designates 74.125.83.41 as permitted sender)
MIME-Version: 1.0
Date: Tue, 29 Nov 2011 15:37:06 +0800
Message-ID: 
 <CANPB7a7VRP8z3ce=jReBB7O2MZZN4xBN3nzW2gSZE7j889Ed2Q@mail.gmail.com>
Subject: Use single cluster or two clusters for log analysis and HBase?
From: jingguo yao <yaojingguo@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=00151749f35cacf52604b2daaf0a

--00151749f35cacf52604b2daaf0a
Content-Type: text/plain; charset=ISO-8859-1

I want to set up Hadoop clusters. There are two workloads. One is log
analysis which is using MapReduce to process big log files in HDFS.
The other is HBase which is used to serve random table queries.

I have two choices to set up my Hadoop clusters. One is to use one
Hadoop cluster. Log analysis and HBase use the same cluster. Its
advantages are:

1 There is only one Hadoop cluster which I need to manage.
2 Both MapReduce and HBase can use this big cluster which has more
  storage and more powerful computation capability.

Its disadvantages:

1 Running MapReduce jobs may slow down the random HBase table
  queries.

The other choice is to use two clusters. Cluster A is for log analysis.
Cluster B is for HBase. Its advantages are:

1 There are no interferences between log analysis and HBase table
  queries.

Its disadvantages:

1. There are two Hadoop clusters which need to be managed.
2. Both log analysis and HBase queries can only use a small Hadoop
   cluster which has less storage and less powerful computation
   capability.

I don't know which choice is better. Can anybody give me some advice
on this? Thanks.

-- 
Jingguo

--00151749f35cacf52604b2daaf0a--