hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Regions loading too fast
Date Fri, 24 Sep 2010 23:31:15 GMT
The '1' is give a regionserver '1' region to open each time it checks
in.  Servers check in every second by default.  It looks like you have
a good few servers coming in.

Change this if you'd have them come in less frequently:


  <property>
    <name>hbase.regionserver.msginterval</name>
    <value>1000</value>
    <description>Interval between messages from the RegionServer to HMaster
    in milliseconds. Use a high value like 3000 for clusters with more than 10
    nodes. Default is 1 second so that HBase seems more 'live'.
    </description>
  </property>

The java.io.EOFException is something else, a bad storefile or an
issue with hdfs.  Check your hdfs logs for xceiver complaint.

St.Ack


On Fri, Sep 24, 2010 at 4:21 PM, Jack Levin <magnito@gmail.com> wrote:
> Also, even with '1' value, I see:
>
> 2010-09-24 16:20:29,983 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> img834,1000351n.jpg,1285251664421.d09510a16c0cfd0d8a251a36229125e0.
> 2010-09-24 16:20:29,984 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> img651,pict1408.jpg,1285018965749.110871465
> 2010-09-24 16:20:29,984 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> img806,sam0084a.jpg,1285324613056.82a1e8ba8d2a37a591a847fb36803c45.
> 2010-09-24 16:20:29,985 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> img535,screenshot1bt.png,1285323376435.fae5f3ab474196c99f10b8a936fb9ead.
> 2010-09-24 16:20:29,985 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> img838,123468.jpg,1285223690165.a2903008621d1a6b6ca02441bf3b68ea.
> 2010-09-24 16:20:29,985 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> img839,yug.jpg,1285230318537.c09323dbaf54130671df2a14d671fe25.
> 2010-09-24 16:20:29,985 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> img821,vlcsnap78737.png,1285283076812.ea4973ce6e43d7f974613c5989647278.
> 2010-09-24 16:20:29,985 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> img805,njt30scbkdmb.gif,1285322429401.f9aacdafd8064bfbcc8cd4f6930febbe.
> 2010-09-24 16:20:29,985 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> img94,img1711m.jpg,1285016850260.1424182007
> 2010-09-24 16:20:29,986 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Creating region
> img840,kitbarca2.png,1285189312696.1ce170ec09384fca51297a5fe7aeb4af.
>
> Which is pretty close to concurrent.
>
> -Jack
>
> On Fri, Sep 24, 2010 at 4:16 PM, Jack Levin <magnito@gmail.com> wrote:
>> Still having a problem:
>>
>> 2010-09-24 16:15:02,572 ERROR
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening
>> img695,p1908101232.jpg,1285288492084.d451f05024b42f71a115951c62cdcccf.
>> java.io.EOFException
>>        at java.io.DataInputStream.readFully(DataInputStream.java:180)
>>        at org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:63)
>>        at org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:101)
>>        at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1937)
>>        at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1837)
>>
>>
>> I changed the value to '1', and restarted the regionserver... Note
>> that my hdfs is not having a problem.
>>
>> -Jack
>>
>> On Fri, Sep 24, 2010 at 4:01 PM, Stack <stack@duboce.net> wrote:
>>> Try
>>>
>>>  <property>
>>>    <name>hbase.regions.percheckin</name>
>>>    <value>10</value>
>>>    <description>Maximum number of regions that can be assigned in a single
go
>>>    to a region server.
>>>    </description>
>>>  </property>
>>>
>>> What do you have now?  Whatever it is, go down from there.
>>>
>>> St.Ack
>>>
>>>
>>> On Fri, Sep 24, 2010 at 3:07 PM, Jack Levin <magnito@gmail.com> wrote:
>>>> My regions are 1gb in size and when I cold start the cluster I oversaturate
my network links (1000 mbps) and get client dfs timeouts , anyway to slow the m down?
>>>>
>>>> -Jack
>>>>
>>>>
>>>
>>
>

Mime
View raw message