hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: 【region compaction check key-bound or not】
Date Thu, 11 Jul 2013 01:09:20 GMT
bq. upgrade to 0.94.6

Nit: upgrade to 0.94.6.1 (or 0.94.9 which was just released)

Cheers

On Wed, Jul 10, 2013 at 6:04 PM, Sergey Shelukhin <sergey@hortonworks.com>wrote:

> Yeah, this is not going to work... moving storefiles manually between
> regions is generally not a good idea.
> Do you want to move entire table into new table? Then the best thing is
> probably to upgrade to 0.94.6 (rolling restart is supported from 0.94.3)
> and use HBase snapshots. You can use export table w/o upgrading.
> If you want to merge into existing table I'm not really sure what is the
> best way to do that. You might have to rewrite data with target region
> boundaries, or manually split and merge regions accordingly and run
> compactions. Then you can disable the table, copy the files and bulk load
> them into target cluster. See
> http://hbase.apache.org/book/arch.bulk.load.html
> Note that bulk load requires boundaries that fit within one region
> (matching or sub-range). There may be a tool that solves this but I'm not
> aware of it...
>
>
> On Wed, Jul 10, 2013 at 5:40 PM, Bing Jiang <jiangbinglover@gmail.com
> >wrote:
>
> > Thanks,Sergey.
> > These days, I want to move a table from one hbase cluster to another
> hbase
> > cluster, and there are the same table's schema.
> > So I want to move a region's storefile to another table's region
> > corresponding directory, and the key bound of regions are overlapped.
> > For example:
> > Cluster   |  Table | Region's key bound
> > cluster1  |  dat    |  [1ffff,2ffff]
> > cluster2  |  dat    |  [2bfff,31fff]
> >
> > In my opinion , if region files in cluster2 are move into cluster1's
> > regions, and make compaction(Minor && Major) upon the first region,
it
> will
> > prune the improper key in (2ffff,31fff].
> > However I found hbase compaction cannot support that.
> >
> > Any idea would be thankful.
> >
> >
> > 2013/7/11 Sergey Shelukhin <sergey@hortonworks.com>
> >
> > > You should not have to manually take care of region bounds in normal
> > > circumstances (unless you are reading the file from coprocessor in some
> > > special way, or something like that). Please tell us if you are seeing
> > any
> > > strange behavior :)
> > > See HalfStoreFileReader for the code that is used to read the
> referenced
> > > file and constrains the keys.
> > >
> > > On Wed, Jul 10, 2013 at 4:25 AM, Bing Jiang <jiangbinglover@gmail.com
> > > >wrote:
> > >
> > > > Hi,all
> > > >
> > > > If Region process splits, it will make a reference.After executing
> > child
> > > > region makes a compaction that absorbs all the reference. And I have
> a
> > > > question that how to make differences when executes scanner.
> > > > As we know that Compaction uses the scanner as well, so whether to
> set
> > > the
> > > > startKey and endKey of child region's bound, in order to guarantee
> that
> > > the
> > > > storefile in child region will not contain outlier keys? I cannot
> find
> > > code
> > > > to prove it, BTW, we use 0.94.3.
> > > >
> > > > Thanks.
> > > >
> > > > --
> > > > Bing Jiang
> > > > weibo: http://weibo.com/jiangbinglover
> > > > BLOG: http://blog.sina.com.cn/jiangbinglover
> > > > National Research Center for Intelligent Computing Systems
> > > > Institute of Computing technology
> > > > Graduate University of Chinese Academy of Science
> > > >
> > >
> >
> >
> >
> > --
> > Bing Jiang
> > Tel:(86)134-2619-1361
> > weibo: http://weibo.com/jiangbinglover
> > BLOG: http://blog.sina.com.cn/jiangbinglover
> > National Research Center for Intelligent Computing Systems
> > Institute of Computing technology
> > Graduate University of Chinese Academy of Science
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message