Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4E4A04B2F for ; Wed, 13 Jul 2011 03:05:18 +0000 (UTC) Received: (qmail 3313 invoked by uid 500); 13 Jul 2011 03:05:16 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 2801 invoked by uid 500); 13 Jul 2011 03:05:02 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 2787 invoked by uid 99); 13 Jul 2011 03:04:56 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Jul 2011 03:04:56 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of saint.ack@gmail.com designates 209.85.216.169 as permitted sender) Received: from [209.85.216.169] (HELO mail-qy0-f169.google.com) (209.85.216.169) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Jul 2011 03:04:48 +0000 Received: by qyk32 with SMTP id 32so81227qyk.14 for ; Tue, 12 Jul 2011 20:04:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type :content-transfer-encoding; bh=2SpSjFNC3fgNDKQ9+bu9Z3Dp8HqfBikm0vBsMcGD+lw=; b=GGY09Oqe0D8rpnneCV95UeP8BFT6QhI+eA5kqRMt1pr+gqYIQtbA8xaZpm7iDLjaFg FuG6f65uDRjt7yaK+ljG2IcwSYWrxJ7645BrmBxWTKn8bBxwSrTr+ztVu7Ps43FtvUMe lysqjivJbu0lr6Dwv95Ej8d8uHYk/JlbQhTCk= MIME-Version: 1.0 Received: by 10.224.193.137 with SMTP id du9mr611082qab.115.1310526267597; Tue, 12 Jul 2011 20:04:27 -0700 (PDT) Sender: saint.ack@gmail.com Received: by 10.224.60.196 with HTTP; Tue, 12 Jul 2011 20:04:27 -0700 (PDT) In-Reply-To: References: <996FE584DE9DE642B04DF887487D18F016B94A4B4F@SP2-EX07VS01.ds.corp.yahoo.com>

Date: Tue, 12 Jul 2011 20:04:27 -0700 X-Google-Sender-Auth: z9_0Mf7kTsNBu0JXcegMM14AzHM Message-ID: Subject: Re: region max filesize From: Stack To: user@hbase.apache.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Check out the copytables tool. See if that will work for you. Here is its usage: stack@sv4borg227:~/hadoop-0.22$ ./bin/hadoop jar ~/hbase-0.91.0-SNAPSHOT/hbase-0.91.0-SNAPSHOT.jar copytable Usage: CopyTable [--rs.class=3DCLASS] [--rs.impl=3DIMPL] [--starttime=3DX] [--endtime=3DY] [--new.name=3DNEW] [--peer.adr=3DADR] Options: rs.class hbase.regionserver.class of the peer cluster specify if different from current cluster rs.impl hbase.regionserver.impl of the peer cluster starttime beginning of the time range without endtime means from starttime to forever endtime end of the time range new.name new table's name peer.adr Address of the peer cluster given in the format hbase.zookeeer.quorum:hbase.zookeeper.client.port:zookeeper.z= node.parent families comma-seperated list of families to copy Args: tablename Name of the table to copy Examples: To copy 'TestTable' to a cluster that uses replication for a 1 hour window= : $ bin/hbase org.apache.hadoop.hbase.mapreduce.CopyTable --rs.class=3Dorg.apache.hadoop.hbase.ipc.ReplicationRegionInterface --rs.impl=3Dorg.apache.hadoop.hbase.regionserver.replication.ReplicationReg= ionServer --starttime=3D1265875194289 --endtime=3D1265878794289 --peer.adr=3Dserver1,server2,server3:2181:/hbase TestTable stack@sv4borg227:~/hadoop-0.22$ St.Ack On Tue, Jul 12, 2011 at 5:17 PM, =E9=99=88=E5=8A=A0=E4=BF=8A wrote: > create simple m/r job to migrate data to new tables and drop the old ones= =E3=80=82 > > > Could you put your code paste to here? > > On Wed, Jul 13, 2011 at 7:43 AM, Ravi Veeramachaneni < > ravi.veeramachaneni@gmail.com> wrote: > >> Albert, >> >> You doing partially right. The change will take into effect only to new >> tables created going forward. There are two options I'm aware of, 1. Use >> alter command on HBase shell to specify new region size. 2. Create new s= et >> of tables and create simple m/r job to migrate data to new tables and dr= op >> the old ones. >> >> We followed the second approach when we bump the region size. The first >> approach leaves the current region count But fills up until region gets = to >> 1GB. Our goal was to reduce number of regions to minimum, second approac= h >> worked well for us. >> >> Thanks, >> Ravi >> >> Sent from my mobile phone. >> >> On Jul 11, 2011, at 8:56 PM, Albert Shau wrote: >> >> > Hi everyone, >> > >> > I'm using 0.90.3 and have the region max filesize set to 1gb in my >> hbase-site.xml like so: >> > >> > =C2=A0 >> > =C2=A0 =C2=A0hbase.hregion.max.filesize >> > =C2=A0 =C2=A01073741824 >> > =C2=A0 >> > >> > But when I look at the storefileSizeMB values listed on the regionserv= er >> UI, they're all around 256mb, which makes me think the default value is >> still being used for whatever reason. =C2=A0I'm also setting other prope= rties >> like the handler count which are getting used so I'm pretty sure I have = the >> conf in the right place and it's getting read. =C2=A0I've also tried res= tarting >> hbase and doing a major compact on the table but the region sizes stay t= he >> same. =C2=A0Any ideas on what might be wrong? =C2=A0Is there some way to= verify the >> max filesize is getting set correctly? >> > >> > Thanks, >> > Albert >> > >> > > > > -- > Thanks & Best regards > jiajun >