hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: WrongRegionException -- add_table.rb screwed up my hbase.
Date Wed, 12 May 2010 05:56:29 GMT
Sorry for trouble caused.  I thought that 0.20.4 added updating of
.regioninfo on renable of table but I don't see it.  Nonetheless, I'd
suggest you update to 0.20.4.  It should have fixes at least to save
your from WRE going forward.

Thanks for writing the list,
St.Ack

On Tue, May 11, 2010 at 9:20 PM, maxjar10 <jcuzens@gmail.com> wrote:
>
> Answered my own question. The .regioninfo files are there specifically for
> performing fsck functionalities like using add_table.rb.
>
> The problem is that the .regioninfo files are NOT updated after an alter.
> This issue is described in:
>
> https://issues.apache.org/jira/browse/HBASE-2366
>
> The purpose of the .regioninfo files is described here:
>
> http://www.larsgeorge.com/2009/10/hbase-architecture-101-storage.html
>
>
> maxjar10 wrote:
>>
>> Ok, here's my story in case anyone else encounters the same issue...
>>
>> My question is this... Why does the table descriptor/meta table
>> information not match the .regioninfo in each region sub dir? Is this a
>> bad thing? Read below...
>>
>> HBase 0.23-1
>> Hadoop 0.20.1
>>
>> So I wanted to add compression to my HBase tables that I already had
>> setup. So, I went to the hbase shell and ran a alter table set compression
>> to GZ and decreased the versions from 3 to 2.
>>
>> I then ran a major_compact on my table to put the change into effect. Even
>> though this appears to happen instantly I know you need to wait for
>> fragmentation to drop to 0%.
>>
>> Now, I wanted to run a job and saw an exception:
>>
>> org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
>> contact region server Some server, retryOnlyOne=true, index=0,
>> islastrow=true, tries=9, numtries=10, i=0, listsize=1,
>> region=Businesses,300120000005:14317197,1269286328987 for region
>> Businesses,300120000005:14317197,1269286328987, row '300120000013:692',
>> but failed after 10 attempts.
>>
>> How very strange... Well, this was odd so I went to HBase shell and ran a
>> count "Businesses" and it hung (looped, whatever) when it got ~ to the
>> above start row. So, I cancelled the count and saw a brief message in the
>> datanode log about the fact that there was a WrongRegionException. Hmmm...
>>
>> When I looked at the .META. tables I saw that the endRow didn't match up
>> to the next startRow like it should so it looked as though a region was
>> missing. Double hmmmm...
>>
>> Because I clearly had something wrong I decided to try to run the
>> add_table.rb as suggested in another thread about seeing
>> WrongRegionExceptions. So, I proceeded to run the add_table.rb only to
>> have it fail. I modified the script and noticed that it was outputting the
>> .regioninfo information that is in each regions sub dir. The problem is
>> that this .regioninfo DOES NOT match the actual table description. There
>> was no COMPRESSION and the version was back at 3. This was confusing
>> because I could see in the actual data when I would open the data files
>> that it was clearly gzipped.
>>
>> Sigh, so, I went ahead and modified the script to output what does match
>> the table description and the only thing it uses the .regioninfo files for
>> is the startKey and the endKey. The rest is the data that actually matches
>> my alter.
>>
>> As of right now the count is proceeding passed the row I had a problem
>> with BUT I'm not sure if the data is actually good. I'll need to scan a
>> few rows.
>>
>> My question is this... Why does the table descriptor/meta table
>> information not match the .regioninfo in each region sub dir? Is this a
>> bad thing?
>>
>> Thanks!
>>
>
> --
> View this message in context: http://old.nabble.com/WrongRegionException----add_table.rb-screwed-up-my-hbase.-tp28531026p28531887.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>

Mime
View raw message