hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: HConnectionImplementation.listTables and list table exception
Date Wed, 23 Oct 2013 17:10:49 GMT
bq. our hbase version was 0.94.0 and hadoop version was 1.0.3

If possible, consider upgrading HBase to 0.94.12 and Hadoop to 1.2.1

Cheers


On Wed, Oct 23, 2013 at 7:18 AM, 张莉苹 <zlpmichelle@gmail.com> wrote:

> Hello Ted, and Julian,
>
> It seemed I didn't receive your mail from my gmail inbox. It was strange.
> Anyhow I found your reply by google search. :)
>
> I'll answer your questions here, and thanks very much for your reply.
>
>
> From *Ted*:
>
> What version of HBase and Hadoop are you using ?
>
> >> our hbase version was 0.94.0 and hadoop version was 1.0.3
>
> Can you show us more of the master log ?
>
> >> I'm so sorry, the master log could not be accessed. The environment was
> only used by me for a short while.
>
>
> From * Michael <
> http://www.mail-archive.com/search?l=user@hbase.apache.org&q=from:%22Michael+Segel%22
> >:*
>
>
> Why 9 zookeepers?
>
> >> the previous zookeeper number was 5, we also thought the number of
> zookeeper was too small, so we increased it into 9, but it still failed.
>
>
> From *Julian*:
>
>
> Hello Michelle,
>    How many regions totally are there in your 600 nodes cluster? Looks
> like many of them are pending for open and being assigned to region
> servers.
> Can you see many items under zookeeper dir /hbase/unassigned?
>
> >> we had 60K regions, the env could not be accessed.
>
> You would like to refer
> http://blog.sina.com.cn/s/blog_4a1f59bf01018tu4.html?
>
> >> kind of help, thanks!
>
>
>
>
>
> Cheers,
> -----
> Big Data - Big Wisdom - Big Value
> --------------
> Michelle Zhang (Li Ping Zhang)
>
>
> 2013/10/23 张莉苹 <zlpmichelle@gmail.com>
>
> > Dear  HBase dev and users,
> >
> > Did you meet this
> >
> "org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.listTables"
> > issue?
> >
> > We setup a 600 nodes cluster, 9 zookeeper nodes to load data into hbase,
> > but it seemed hbase master was busy handling transition with zookeeper,
> > and hbase “list” could not get response. The hbase table was created but
> > it didn't  do any insert.
> >
> > Do you have any idea of  the root cause and how to fix it?  :)Highly
> > appreciate for your answers!
> >
> >
> >
> > Here is the exception stack:
> > ---------------------------------------------------
> > java.lang.reflect.UndeclaredThrowableException
> >                at $Proxy7.getHTableDescriptors(Unknown Source)
> >                at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.listTables(HConnectionManager.java:2237)
> >                at
> > org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:317)
> >
> >
> >
> >
> > hbase master log:
> >
> > -----------------------------
> >
> > 2013-10-18 06:19:41,279 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> > master:60000-0x341be88202300ab* Deleting existing unassigned node* for
> > 0ec3308bd1e2bdd9576b2d60d2eee68e that is in expected state
> > RS_ZK_REGION_OPENED
> >
> > 2013-10-18 06:19:41,279 DEBUG
> > org.apache.hadoop.hbase.master.AssignmentManager:* Handling
> > transition=RS_ZK_REGION_OPENING*, s*erver=node0878*.
> > ic.analyticsworkbench.com,60020,1381883086785,
> > region=15a4fb29aa1d905b13f33594e50bc8de, which is more than 15 seconds
> late
> >
> > 2013-10-18 06:19:41,280 DEBUG
> > org.apache.hadoop.hbase.master.AssignmentManager: *Handling
> > transition=RS_ZK_REGION_OPENING, server=node0898*.
> > ic.analyticsworkbench.com,60020,1381883200494,
> > region=1a4c929534e6828c85f22b062f949304, which is more than 15 seconds
> late
> >
> > 2013-10-18 06:19:41,289 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> > master:60000-0x341be88202300ab Successfully *deleted unassigned node *for
> > region 0ec3308bd1e2bdd9576b2d60d2eee68e in expected state
> > RS_ZK_REGION_OPENED
> >
> > 2013-10-18 06:19:41,289 DEBUG
> > org.apache.hadoop.hbase.master.AssignmentManager: Handling
> > transition=RS_ZK_REGION_OPENING, server=
> node0693.ic.analyticsworkbench.com,60020,1381881773670,
> > region=d47bfe1af0051c405de295a51c1c6e63, which is more than 15 seconds
> late
> >
> >
> >
> > We also try to "list" in hbase shell,it also failed:
> >
> > The hbase “list” got error as:
> >
> > ------------------------------------------
> >
> >
> >
> > hbase(main):001:0> list
> >
> > TABLE
> >
> >
> >
> >
> > ERROR: java.lang.reflect.UndeclaredThrowableException: Call to
> > node0997.ic.analyticsworkbench.com/10.1.50.17:60000 failed on socket
> > timeout exception: java.net.SocketTimeoutException: 120000 millis timeout
> > while waiting for channel to be ready for read. ch :
> > java.nio.channels.SocketChannel[connected local=/10.1.50.15:45726remote=
> > node0997.ic.analyticsworkbench.com/10.1.50.17:60000]
> >
> >
> >
> >
> > Cheers,
> > -----
> > Big Data - Big Wisdom - Big Value
> > --------------
> > Michelle Zhang (Li Ping Zhang)
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message