Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of yuzhihong@gmail.com
 designates 209.85.161.41 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=YgFZzXwqBq5uMPaFHr7YHlcCje2QdlVd0gS2TCBLvFcjab/zPSg1y8lMK4zW7FELT6
         +5wh/IHSCb7BAV9LG7sjmo6AvYFNQ4B5dt81gkA3DVKlnN3fzHQgsUPRHFQ4q4HP0b1y
         w4KatrHlq3T6vvp1T5Wi9k7mXYimQxSrpdmQo=
MIME-Version: 1.0
In-Reply-To: 
 <853846865-1291677394-cardhu_decombobulator_blackberry.rim.net-1639593731-@bda2307.bisx.prod.on.blackberry>
References: <16616384.116971291560610632.JavaMail.jira@thor>
	<1211721.2551291662087461.JavaMail.jira@thor>
	<853846865-1291677394-cardhu_decombobulator_blackberry.rim.net-1639593731-@bda2307.bisx.prod.on.blackberry>
Date: Tue, 7 Dec 2010 10:16:06 -0800
Message-ID: <AANLkTimOk+kopXVFPETr-q0TvWzwQ8=TAGLfVarT9h65@mail.gmail.com>
Subject: Re: [jira] Commented: (HBASE-3312) Region fails to split
From: Ted Yu <yuzhihong@gmail.com>
To: dev@hbase.apache.org, apurtell@apache.org
Content-Type: multipart/alternative; boundary=20cf3054a8cf87fcbd0496d5ff34

--20cf3054a8cf87fcbd0496d5ff34
Content-Type: text/plain; charset=ISO-8859-1

Andrew:
I guess you don't see the following (from line 1356 in Store.checkSplit())
in your log:
          if (LOG.isDebugEnabled()) {
            LOG.debug("cannot split because midkey is the same as first or "
+
              "last row");
          }
Was it possible that the largest StoreFile grew (symmetrically) on both
sides of the midkey (which happened to be the start row) ?

On Mon, Dec 6, 2010 at 3:16 PM, Andrew Purtell <apurtell@apache.org> wrote:

> I'm catching a flight to Narita, then another to Los Angeles today, and
> will look at this when I get home.
>
> Just so you don't think I dropped this on jira and ran off. :-)
>
> I also have that OOME issue to look at also, and I wonder if this split
> problem is related.
>
>     - Andy
>
> -----Original Message-----
> From: "stack (JIRA)" <jira@apache.org>
> Date: Mon, 6 Dec 2010 14:01:27
> To: <apurtell@apache.org>
> Subject: [jira] Commented: (HBASE-3312) Region fails to split
>
>
>    [
> https://issues.apache.org/jira/browse/HBASE-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12968365#action_12968365]
>
> stack commented on HBASE-3312:
> ------------------------------
>
> This is the formatter for the message you see:
>
> {code}
>      LOG.info("Split row is not inside region key range or is equal to " +
>          "startkey: " + Bytes.toString(this.splitrow));
> {code}
>
> So, when it logs:
>
> {code}
> Split row is not inside region key range or is equal to startkey:
> DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF
> {code}
>
> ... we are actually printing out the proposed 'splitrow' which indeed is
> same as the start row.
>
> Why do we keep picking the same splitrow over and over though region is
> large?  Must be something dumb we're doing w/ mutliple families or so?
>
> Good on you Andrew.
>
>
>
> > Region fails to split
> > ---------------------
> >
> >                 Key: HBASE-3312
> >                 URL: https://issues.apache.org/jira/browse/HBASE-3312
> >             Project: HBase
> >          Issue Type: Bug
> >            Reporter: Andrew Purtell
> >            Priority: Critical
> >             Fix For: 0.90.0
> >
> >
> > "SplitTransaction: Split row is not inside region key range or is equal
> to startkey".
> > I stopped writers after realizing one region of the table was growing
> unbounded:
> > {noformat}
> > webtable,DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF,1291557809928.
> > deccb20bcbf8e634008cf093105c4fc5.
> >             stores=3, storefiles=10, storefileSizeMB=6753,
> memstoreSizeMB=2,
> >  storefileIndexSizeMB=2
> > {noformat}
> > In the regionserver log, every compaction of this region fails to split
> with the following message:
> > {noformat}
> > 2010-12-05 09:04:50,156 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> > Split row is not inside region key range or is equal to startkey:
> > DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF
> > {noformat}
> > However there are many rows in the region:
> > {noformat}
> > 10/12/05 09:33:33 DEBUG client.HTable$ClientScanner: Advancing internal
> scanner
> > to startKey at 'DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF'
> > [...]
> > Current count: 258000, row: DE424FBDBD15FF3B3E9D0C3DB149ECD29B0F615B
> > Current count: 259000, row: DF27251479D6C91B27AA9B1561070A53011A6D1E
> > 10/12/05 09:33:36 DEBUG client.HTable$ClientScanner: Finished with region
> REGION =>
> > {NAME => 'webtable,DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF,
> > 1291557809928.deccb20bcbf8e634008cf093105c4fc5.',
> > STARTKEY => 'DE0CBA1D6CDFCDD6CBC1065D2C9C1CA17BDA0FAF',
> > ENDKEY => 'DF76CF458433DB5D0CB2C50042452B296E3721A7',
> > ENCODED => deccb20bcbf8e634008cf093105c4fc5, TABLE => {{NAME =>
> 'webtable',
> > FAMILIES => [{NAME => 'content', BLOOMFILTER => 'NONE', REPLICATION_SCOPE
> => '0',
> > VERSIONS => '2147483647', COMPRESSION => 'LZO', TTL => '2147483647',
> > BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'},
> > {NAME => 'info', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
> > VERSIONS => '2147483647', COMPRESSION => 'LZO', TTL => '2147483647',
> > BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'},
> > {NAME => 'url', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0',
> > VERSIONS => '1', COMPRESSION => 'LZO', TTL => '2147483647',
> > BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'}]}}
> > 10/12/05 09:33:36 DEBUG client.HTable$ClientScanner: Advancing internal
> scanner
> > to startKey at 'DF76CF458433DB5D0CB2C50042452B296E3721A7'
> > {noformat}
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>

--20cf3054a8cf87fcbd0496d5ff34--