Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of doug.meil@explorysmedical.com
 designates 216.32.180.188 as permitted sender)
Received-SPF: pass (mail40-co1: domain of explorysmedical.com designates
 157.56.241.133 as permitted sender) client-ip=157.56.241.133;
 envelope-from=doug.meil@explorysmedical.com;
 helo=BL2PRD0411HT003.namprd04.prod.outlook.com ;.outlook.com ;
From: Doug Meil <doug.meil@explorysmedical.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
CC: "stack@duboce.net" <stack@duboce.net>
Subject: Re: how client location a region/tablet?
Thread-Topic: how client location a region/tablet?
Thread-Index: AQHNfSHWcDciiuFrD06kHfTNh8Ln2pdgj1UAgAB0IgCABhLYAA==
Date: Thu, 23 Aug 2012 12:21:31 +0000
Message-ID: <CC5B986F.4B7AE%doug.meil@explorysmedical.com>
In-Reply-To: 
 <CAK_MoSuGs8qmx5wJGZDvVKmBbYHa5U_E2OCCDUpNs8B-mEMt+w@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="Windows-1252"
Content-ID: <A3C6DD59B56BF14BBFA822E36457544D@namprd04.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0


For further information about the catalog tables and region-regionserver
assignment, see this=8A

http://hbase.apache.org/book.html#arch.catalog


On 8/19/12 7:36 AM, "Lin Ma" <linlma@gmail.com> wrote:

>Thank you Stack, especially for the smart 6 round trip guess for the
>puzzle. :-)
>
>1. "Yeah, we client cache's locations, not the data." -- does it mean for
>each client, it will cache all location information of a HBase cluster,
>i.e. which physical server owns which region? Supposing each region has
>128M bytes, for a big cluster (P-bytes level), total data size / 128M is
>not a trivial number, not sure if any overhead to client?
>2. A bit confused by what do you mean "not the data"? For the client
>cached
>location information, it should be the data in table METADATA, which is
>region / physical server mapping data. Why you say not data (do you mean
>real content in each region)?
>
>regards,
>Lin
>
>On Sun, Aug 19, 2012 at 12:40 PM, Stack <stack@duboce.net> wrote:
>
>> On Sat, Aug 18, 2012 at 2:13 AM, Lin Ma <linlma@gmail.com> wrote:
>> > Hello guys,
>> >
>> > I am referencing the Big Table paper about how a client locates a
>>tablet.
>> > In section 5.1 Tablet location, it is mentioned that client will cache
>> all
>> > tablet locations, I think it means client will cache root tablet in
>> > METADATA table, and all other tablets in METADATA table (which means
>> client
>> > cache the whole METADATA table?). My question is, whether HBase
>> implements
>> > in the same or similar way? My concern or confusion is, supposing each
>> > tablet or region file is 128M bytes, it will be very huge space (i.e.
>> > memory footprint) for each client to cache all tablets or region
>>files of
>> > METADATA table. Is it doable or feasible in real HBase clusters?
>>Thanks.
>> >
>>
>> Yeah, we client cache's locations, not the data.
>>
>>
>> > BTW: another confusion from me is in the paper of Big Table section
>>5.1
>> > Tablet location, it is mentioned that "If the client=B9s cache is stal=
e,
>> the
>> > location algorithm could take up to six round-trips, because stale
>>cache
>> > entries are only discovered upon misses (assuming that METADATA
>>tablets
>> do
>> > not move very frequently).", I do not know how the 6 times round trip
>> time
>> > is calculated, if anyone could answer this puzzle, it will be great.
>>:-)
>> >
>>
>> I'm not sure what the 6 is about either.  Here is a guesstimate:
>>
>> 1. Go to cached location for a server for a particular user region,
>> but server says that it does not have a region, the client location is
>> stale
>> 2. Go back to client cached meta region that holds user region w/ row
>> we want, but its location is stale.
>> 3. Go to root location, to find new location of meta, but the root
>> location has moved.... what the client has is stale
>> 4. Find new root location and do lookup of meta region location
>> 5. Go to meta region location to find new user region
>> 6. Go to server w/ user region
>>
>> St.Ack
>>