hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy" <sa...@pearsonwholesale.com>
Subject Re: hbase slits 2 null ending keys
Date Fri, 23 Nov 2007 19:39:52 GMT
Thansk for that info. I started to get these messages on restart of hbase so 
I will add data and see if they split. I still have not got any more 
messages or errors that will kill the database but still working on it :) .

Billy



"stack" <stack@duboce.net> wrote in message 
news:47472A5A.60403@duboce.net...
> Billy wrote:
>> I tried to lower my max table size from the 64MB default to 16MB and this 
>> is the errors I am getting now
>>
>
> Do you mean region size in the above?
>
>> 2007-11-22 10:32:05,700 WARN org.apache.hadoop.hbase.HRegion: Region 
>> webdata,jp.co.infoseek.hp.ahodori/:http,1195746881040 is NOT splitable 
>> though its aggregate size is 34.2m and desired size is 16.0m
>> 2007-11-22 10:32:05,706 WARN org.apache.hadoop.hbase.HRegion: Region 
>> webdata,org.btvshalom.bringthemhome/:http,1195746881041 is NOT splitable 
>> though its aggregate size is 34.2m and desired size is 16.0m
>>
>> None of the tables parts will split any more.
>>
>>
>
> The way its supposed to work is that when a region gets 'too big', it 
> splits.  Splits are near-instantaneous because the split daughter regions 
> are manufactured holding 'references' to the parent; one reads from the 
> top half of the parent data files and the other daughter from the lower 
> half.  While a daughter holds references to a parent, it cannot be split 
> (The message you are seeing above it that the region has exceeded maximum 
> size so the splitter thread wants to split the region, but on examination, 
> it aborts because it notices references are still held).  References are 
> cleaned up by compactions.  Compactions are triggered by a region having 
> 'too many' data file instances (HStoreFiles).  The default limit is 3. 
> If you continue to add data, new dumps of the memcache should create new 
> HStoreFile instances which should eventually put you over the compaction 
> threshold so a compaction runs which will make the region splittable, etc.
>
> St.Ack
>
>> Billy
>>
>>
>> "Billy" <sales@pearsonwholesale.com> wrote 
>> in message news:fhvks3$gi8$1@ger.gmane.org...
>>
>>> I ma not sure if I have the logs with the EOF in them I thank I delete 
>>> them when I started over but I will take a look and add an issue with 
>>> what I got.
>>>
>>>
>>> Billy
>>>
>>> "stack" <stack@duboce.net> wrote in message 
>>> news:474355D5.70503@duboce.net...
>>>
>>>> Hey Billy.
>>>>
>>>> Looks like we need to make it so the regionserver show more info on 
>>>> regions (smile).  The first and last key in a table is null.   Your jpg 
>>>> is showing two regions with the null end key.  Thats kinda odd.  Want 
>>>> to make an issue and add logs?  I'll take a look (Include the EOF 
>>>> exception you've been seeing).
>>>>
>>>> Thanks,
>>>> St.Ack
>>>>
>>>> Billy wrote:
>>>>
>>>>> I got a table that was doing good then it split and started getting 
>>>>> EOF exceptions so I delete the database and started over and now its

>>>>> happened again from the regional server GUI it shows this image
>>>>>
>>>>> http://www.pearsonwholesale.com/image2.jpg
>>>>>
>>>>> the table has two part that have nulls at as the ending keys it still

>>>>> loading and running but I betting I will at some point be getting the

>>>>> EOF exception again but I will wait and see.
>>>>>
>>>>> figured this looks like a bug to me but maybe just the regional server

>>>>> being lazy about reporting its end key.
>>>>>
>>>>> let me know if you need logs from the splits.
>>>>>
>>>>> Billy
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
> 




Mime
View raw message