hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1121) Cluster confused about where -ROOT- is
Date Sun, 11 Jan 2009 23:38:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12662840#action_12662840
] 

stack commented on HBASE-1121:
------------------------------

It looks like its the regionhistorian trying to update about a compaction that is failing:

{code}
2009-01-11 06:11:56,405 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG org.apache.hadoop.hbase.regionserver.HStore:
moving /hbasetrunk2/TestTable/compaction.dir/1701213566/info/mapfiles/4916277877406001197
to /hbasetrunk2/TestTable/1701213566/info/mapfiles/3125166216229916041
2009-01-11 06:11:56,472 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG org.apache.hadoop.hbase.regionserver.HStore:
Completed major compaction of 1701213566/info store size is 188.2m
2009-01-11 06:11:56,480 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] INFO org.apache.hadoop.hbase.regionserver.HRegion:
compaction completed on region TestTable,0008454525,1231561159116 in 1mins, 8sec
2009-01-11 06:11:56,484 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Attempt 0 of 10 failed with <org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException:
-ROOT-,,0
        at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2064)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1545)
        at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:894)
>. Retrying after sleep of 2000
2009-01-11 06:11:56,487 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Found ROOT REGION => {NAME => '-ROOT-,,0', STARTKEY => '', ENDKEY => '', ENCODED
=> 70236052, TABLE => {{NAME => '-ROOT-', IS_ROOT => 'true', IS_META => 'true',
FAMILIES => [{NAME => 'info', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS
=> '10', LENGTH => '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE
=> 'true'}], INDEXES => []}}
2009-01-11 06:11:58,491 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Attempt 1 of 10 failed with <org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException:
-ROOT-,,0
        at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2064)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1545)
        at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:894)
>. Retrying after sleep of 2000
...
{code}

Above goes on till attempt 8, then it starts over again at 0 and just keeps on going.

Why ain't we recalibrating to find new -ROOT- location.

Cluster is hosed unless I restart a regionserver one at a time or just restart whole cluster.

> Cluster confused about where -ROOT- is
> --------------------------------------
>
>                 Key: HBASE-1121
>                 URL: https://issues.apache.org/jira/browse/HBASE-1121
>             Project: Hadoop HBase
>          Issue Type: Bug
>         Environment: TRUNK trying to do big upload
>            Reporter: stack
>
> Each node in cluster is doing below:
> {code}
> 2009-01-11 20:52:22,739 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Attempt 0 of 10 failed with <org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException:
-ROOT-,,0
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2064)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1545)
>         at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:894)
> >. Retrying after sleep of 20002009-01-11 20:52:22,742 [regionserver/0:0:0:0:0:0:0:0:60020.compactor]
DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Found ROOT REGION =>
{NAME => '-ROOT-,,0', STARTKEY => '', ENDKEY => '', ENCODED => 70236052, TABLE
=> {{NAME => '-ROOT-', IS_ROOT => 'true', IS_META => 'true', FAMILIES => [{NAME
=> 'info', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '10', LENGTH
=> '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'true'}], INDEXES
=> []}}2009-01-11 20:52:24,759 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Attempt 1 of 10 failed with <org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException:
-ROOT-,,0
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2064)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1545)
>         at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:894)
> {code}
> I tried to close the root region and tripped over hbase-1120.
> So I killed the regionserver hosting -ROOT- but weirdly had no effect as though the 
regionservers' internal cache of -ROOT- location cannot be changed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message