Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of doug.meil@explorysmedical.com
 designates 65.55.88.14 as permitted sender)
Received-SPF: pass (mail151-tx2: domain of explorysmedical.com designates
 157.56.244.181 as permitted sender) client-ip=157.56.244.181;
 envelope-from=doug.meil@explorysmedical.com;
 helo=CH1PRD0410HT002.namprd04.prod.outlook.com ;.outlook.com ;
From: Doug Meil <doug.meil@explorysmedical.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
CC: "hbase-user@hadoop.apache.org" <hbase-user@hadoop.apache.org>
Subject: Re: Get the list of store/store files for a region via HBase API
Thread-Topic: Get the list of store/store files for a region via HBase API
Thread-Index: AQHNMt8OWQmt9OXHSkmGvnNh3kpCtpbLGTIAgABG9oCAAAf6AIABMjKA///9LoA=
Date: Wed, 16 May 2012 20:12:06 +0000
Message-ID: <CBD9811D.2C405%doug.meil@explorysmedical.com>
In-Reply-To: 
 <CAGF+3rb_jGiptFJvaS760n6edm=Oo+HzOcgni-7Yypx6YG1yyw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <5DFE8B219F54C74B90FCA0C8F05F1703@namprd04.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0


With respect to #2, it's more than that.  StoreFiles get written on
MemStore flushes but also re-written whenever compactions happen.  So
regardless of where you get this information, you have to expect that this
answer is going to change.


On 5/16/12 12:22 PM, "Chen Song" <chen.song.82@gmail.com> wrote:

>In HBase API, there are classes defined on each level of structs. For
>example, HRegionInterface, HRegionInfo (HRegion), Store, StoreFile. I am
>not sure why there doesn't seem to have a clear way to traverse this
>hierarchical structure.
>
>I can think of some downsides of using HDFS to get such information.
>
>1. The user has to rebuild and maintain a similar hierarchy of struct
>classes on their end from the HDFS structure. For simple use case such as
>getting counts, it is probably OK but it could enforce the user to
>replicate the similar amount of work (which has been done on HBase API)
>for
>use cases that need more complicated meta information.
>2. If a region is removed from meta store, there exists a timing between
>the time it is removed from meta store and that it is physically removed
>from HDFS. That said, it is not reliable.
>
>Chen
>
>On Tue, May 15, 2012 at 10:06 PM, Doug Meil
><doug.meil@explorysmedical.com>wrote:
>
>>
>> You're just doing a directory listing in HDFS to get this information.
>> That's a pretty lightweight operation (I.e., as opposed to transferring
>> the contents of all the StoreFiles, etc.)
>>
>> If you want to go all the way down to the StoreFile it's the only way
>>I'm
>> aware of at this time.
>>
>>
>>
>>
>> On 5/15/12 5:37 PM, "Chen Song" <chen.song.82@gmail.com> wrote:
>>
>> >Thanks Doug, that should work as the hierarchy is explicitly reflected
>>on
>> >HDFS. But is this the preferred way to do such table/region/storefile
>> >traversal?
>> >
>> >I would like to avoid hitting to HDFS directly if possible as these
>>pieces
>> >of meta information is within HBase world.
>> >
>> >Thanks
>> >Chen
>> >
>> >On Tue, May 15, 2012 at 5:23 PM, Doug Meil
>> ><doug.meil@explorysmedical.com>wrote:
>> >
>> >>
>> >> You can get the Table->Region->StoreFile information via HDFS.  That
>>is
>> >> described here in the RefGuide:
>> >>
>> >> http://hbase.apache.org/book.html#trouble.namenode
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On 5/15/12 5:09 PM, "Chen Song" <chen.song.82@gmail.com> wrote:
>> >>
>> >> >I am new to HBase and started working on a project which needs meta
>> >> >information on HBase regions for a table. The version of HBase I am
>> >>using
>> >> >0.90.4.
>> >> >
>> >> >The use case is very simple.
>> >> >
>> >> >First, I want to get all regions for a table, which I can achieve
>>using
>> >> >the
>> >> >API call below.
>> >> >
>> >> >    HTable table =3D new HTable(conf, tableName);
>> >> >
>> >> >    Map<HRegionInfo, HServerAddress> regions =3D
>>table.getRegionsInfo();
>> >> >
>> >> >
>> >> >Second, I want to get the list of stores (and then store files) for
>> >>each
>> >> >region. This is where I stuck as I could not find a way to do it by
>> >> >searching in the API. It seems that HRegionInterface started
>>supporting
>> >> >API
>> >> >call to retrive the list of store files since 0.95-SNAPSHOT but I
>>don't
>> >> >want to upgrade my HBase version. Below is how to get the
>>corresponding
>> >> >HResionInterface.
>> >> >
>> >> >    HConnection connection =3D table.getConnection();
>> >> >
>> >> >    HRegionInterface regionInterface =3D
>> >> >connection.getHRegionConnection(regionAddress);
>> >> >
>> >> >The series of objects I would like to get in sequence is like:
>> >>HRegionInfo
>> >> >-> HRegion -> Store/StoreFile/HFile. If some can help me with it,
>>that
>> >> >would be great.
>> >> >
>> >> >Thanks very much,
>> >> >
>> >> >Chen
>> >>
>> >>
>> >>
>> >
>> >
>> >--
>> >Chen Song
>> >Mobile: 518-445-5096
>>
>>
>>
>
>
>--=20
>Chen Song
>Mobile: 518-445-5096