Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <571561.45882.qm@web65502.mail.ac4.yahoo.com>
Date: Wed, 29 Apr 2009 06:29:40 -0700 (PDT)
From: Andrew Purtell <apurtell@apache.org>
Reply-To: apurtell@apache.org
Subject: Re: Force split
To: hbase-user@hadoop.apache.org
In-Reply-To: <eb4706e0904290255v172a016egd69875d14902f308@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable


Hi Edward. You still have DFS issues. Have you
followed the DFS related troubleshooting tips on the
troubleshooting page of the HBase wiki? Have you
considered adding more data nodes?

   - Andy

> From: Edward J. Yoon=20
> Subject: Re: Force split
> To: hbase-user@hadoop.apache.org
> Date: Wednesday, April 29, 2009, 2:55 AM
> I found below error message. :(
>=20
> 2009-04-29 16:16:57,380 WARN
> org.apache.hadoop.hdfs.DFSClient: Could
> not get block locations. Aborting...
> 2009-04-29 16:16:57,390 ERROR
> org.apache.hadoop.hbase.regionserver.CompactSplitThread:
> Compaction/Split failed for region
> SparseMatrix_randufgse,000000000002519,1240989321368
> java.io.IOException: Could not read from stream
>         at
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:119)
>         at
> java.io.DataInputStream.readByte(DataInputStream.java:248)
>         at
> org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:325)
>         at
> org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:346)
>         at
> org.apache.hadoop.io.Text.readString(Text.java:400)
>         at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.createBlockOutputStream(=
DFSClient.java:2779)
>         at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DF=
SClient.java:2704)
>         at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.ja=
va:1997)
>         at
> org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClie=
nt.java:2183)
> 2009-04-29 16:18:46,051 INFO
> org.apache.hadoop.io.compress.CodecPool:
> Got brand-new decompressor
> 2009-04-29 16:18:46,051 INFO
> org.apache.hadoop.io.compress.CodecPool:
> Got brand-new decompressor
>=20
>=20
> On Wed, Apr 29, 2009 at 6:17 PM, Edward J. Yoon
> <edwardyoon@apache.org> wrote:
> > According to my understanding, there is no any code
> for
> > MODIFY_TABLE_SPLIT and table.jsp work as below.
> >
> > Writable[] arr =3D new Writable[1];
> > arr[0] =3D new
> ImmutableBytesWritable(Bytes.toBytes(key));
> > master.modifyTable(Bytes.toBytes(tableName),
> > HConstants.MODIFY_TABLE_SPLIT, arr);
> >
> > Anyway, I'd like to know how to force split.
> > Below is the info of 'Regions in randufgse'
> page.
> > Rows are sequntial numbers between 0 and 10,000.
> >
> > If I want to split by "000000000007648"
> then,
> >
> > master.modifyTable(Bytes.toBytes(tableName),
> > HConstants.MODIFY_TABLE_SPLIT, new
> >
> ImmutableBytesWritable(Bytes.toBytes("000000000007648"));
> >
> > Is enough?
> >
> > d8g055.nhncorp.com:60020: =A0, 000000000000465
> > a50891.nhncorp.com:60020: 000000000000465,
> 000000000000811
> > a51019.nhncorp.com:60020: 000000000000811,
> 000000000001134
> > a51019.nhncorp.com:60020: 000000000001134,
> 000000000001460
> > a51010.nhncorp.com:60020: 000000000001460,
> 000000000001723
> > a51010.nhncorp.com:60020: 000000000001723,
> 000000000001988
> > a51018.nhncorp.com:60020: 000000000001988,
> 000000000002254
> > a51018.nhncorp.com:60020: 000000000002254,
> 000000000002519
> > a50837.nhncorp.com:60020: 000000000002519,
> 000000000004648
> > a50837.nhncorp.com:60020: 000000000004648,
> >
> > On Wed, Apr 29, 2009 at 5:11 PM, Ryan Rawson
> <ryanobjc@gmail.com> wrote:
> >> It works, it has worked in the past...
> >>
> >> One thing to remember is you can't always
> split... =A0You cant split until all
> >> the open scanners are closed. =A0Furthermore you
> can't split if a region has
> >> just split. =A0The reason for this is because once
> you split a region, files
> >> aren't rewritten, but the 2 daughter regions
> hold references to the parent
> >> region. =A0Until those regions are resolved via
> compaction (slow, minutes
> >> possibly), you can't split those daughter
> regions.
> >>
> >> So check again, maybe your splits are working
> after all.
> >>
> >> If you are doing imports, check out my
> Randomize.java i posted to the list
> >> last week. =A0That helps import performance
> substantially.
> >>
> >> On Wed, Apr 29, 2009 at 12:52 AM, Edward J. Yoon
> <edwardyoon@apache.org>wrote:
> >>
> >>> Hi,
> >>>
> >>> I saw that function on the web UI but it seems
> not implemented yet.
> >>>
> >>> public void modifyTable(final byte[]
> tableName, int op, Writable[] args)
> >>> =A0switch (op) {
> >>> =A0 =A0...
> >>> =A0 =A0case MODIFY_TABLE_SPLIT:
> >>> =A0 =A0case MODIFY_TABLE_COMPACT:
> >>> =A0 =A0..
> >>> =A0}
> >>> }
> >>>
> >>> Am I right? Pls let me know if i'm wrong.
> >>> --
> >>> Best Regards, Edward J. Yoon @ NHN, corp.
> >>> edwardyoon@apache.org
> >>> http://blog.udanax.org
> >>>
> >>
> >
> >
> >
> > --
> > Best Regards, Edward J. Yoon @ NHN, corp.
> > edwardyoon@apache.org
> > http://blog.udanax.org
> >
>=20
>=20
>=20
> --=20
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org=0A=0A=0A