hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: hbase regions reporting multiple times
Date Sat, 10 Jul 2010 03:48:58 GMT
Others will have to chime in for details, but typically this means you
are having DNS issues.  That is the hostname is resolving to an ip and
not resolving back to the same name or vice versa or any other combo
of non-roundtripping involving ip and dns names.

-ryan

On Fri, Jul 9, 2010 at 6:41 PM, Arun Ramakrishnan
<aramakrishnan@languageweaver.com> wrote:
> I shutdown hbase. Added some new nodes to hdfs, rebalanced. Also added those nodes to
hbase regionservers.
> Then started hbase.
>
> I am having this strange problem where the new nodes let's say host1 thru host4 gets
repeatedly reported/added to the regionservers list.
>
> Initially when I did a "report 'simple'" from the shell, it showed me 10 unique hosts.
Then within a matter of minutes it grew to 17 ( with the newly added hosts repeating multiple
times).
>
> Also, the web UI failed with the following error.
>
> ##############
> HTTP ERROR: 500
> Trying to contact region server 192.168.130.63:60020 for region .META.,,1, row '', but
failed after 3 attempts.
> Exceptions:
> org.apache.hadoop.hbase.NotServingRegionException: org.apache.hadoop.hbase.NotServingRegionException:
.META.,,1
>        at org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2266)
>        at org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1845)
>        at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
>        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
> ###############
>
>
> Any insight into why the regions get repeated multiple times. I did a  hadoop fsck /
and it reports that all the blocks have been replicated 3 times ( the configured value ).
>
>
> Thanks
> Arun
>

Mime
View raw message