Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: unknown (athena.apache.org: error in processing during lookup of
 mingma@ebay.com)
DomainKey-Signature: s=corp; d=ebay.com; c=nofws; q=dns;
  h=X-EBay-Corp:X-IronPort-AV:Received:Received:From:To:Date:
   Subject:Thread-Topic:Thread-Index:Message-ID:
   Accept-Language:Content-Language:X-MS-Has-Attach:
   X-MS-TNEF-Correlator:acceptlanguage:x-ems-proccessed:
   x-ems-stamp:Content-Type:MIME-Version:X-CFilter;
  b=v+h7qvMHqRmWWonIJ3CEF+hlCLmOzdWoRB544n0bAjiOt6ftzvwthia3
   O/fz8lCR2l8Fih0tyiv/yxl6MoeI5pIdaGQwuZ5qdiEg7xMytoQabyeZa
   bQOYRi7Q1cZ1aoW;
From: "Ma, Ming" <mingma@ebay.com>
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Date: Fri, 2 Sep 2011 16:48:03 -0600
Subject: region transition trace
Thread-Topic: region transition trace
Thread-Index: Acxpwl+0zJbHE6BqTROR9UZ0JPcgSQ==
Message-ID: 
 <D10C58C9BA49DC40AA68056049009BF36464A0D57B@DEN-MEXMS-001.corp.ebay.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: multipart/alternative;
	boundary="_000_D10C58C9BA49DC40AA68056049009BF36464A0D57BDENMEXMS001co_"
MIME-Version: 1.0

--_000_D10C58C9BA49DC40AA68056049009BF36464A0D57BDENMEXMS001co_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Perhaps this feature was discussed before. we currently don't have the full=
 transition history of all regions { fromState, ToState, the machine that i=
nitiates it }; the states will be removed from zookeeper /hbase/unassigned =
once the region is assigned; also ".META." table has max version of 10 thus=
 only tracks the last 10 RS assignments of a given region.

One idea is to put such transition history data to zookeeper. One issue is =
it could blow up zookeeper memory if we have large number of regions and th=
e cluster runs for a long time. I would like to get your feedback on differ=
ent approaches to address the issue. One assumption is region assignment do=
esn't happen with high frequency and thus the overhead introduced won't hav=
e much impact on the system performance.

Approach 1:

Zookeeper knows the history of how /hbase/unassigned is modified, if we can=
 get zookeeper's logs (Bookkeeper ? ) somehow, we know the history of regio=
n transition.

Approach 2:

1.      HBase logs extra region transition data to zookeeper. It could be o=
ne zookeeper node per transaction.
2.      Have a separate thread on the Master to move data from zookeeper an=
d append to HDFS. That will keep the zookeeper size in check.
3.      Have some tool or web UI to show the history of a given region by l=
ooking at zookeeper and HDFS.

Ming


--_000_D10C58C9BA49DC40AA68056049009BF36464A0D57BDENMEXMS001co_--