hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julian Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-8533) [REST] HBaseAdmin does not ride over cluster restart
Date Thu, 16 May 2013 09:37:16 GMT

     [ https://issues.apache.org/jira/browse/HBASE-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Julian Zhou updated HBASE-8533:

    Attachment: hbase-8533-trunk-v0.patch

Using rest client to recreate, there are 2 scenarios, steps could be:
-- Scenario A: request after cluster restart
1) start cluster;
2) start rest at a port 8000;
3) stop the whole cluster (including zk);
4) start the whole cluster again;
5) request http://<ip>:8000/ to get table list. ==> waiting for 10 getMaster retry
(HConnectionManager), each retry with 3 zk connection retries by RecoverableZooKeeper. 10
getMaster retry failed, then "Unavailable" as rest response;

-- Scenario B: rest request between stop and start cluster
1) start cluster;
2) start rest at a port 8000;
3) stop the whole cluster (including zk);
4) request http://<ip>:8000/ to get table list ==> same retry logic as A..
5) start the whole cluster again. ==> rest in 4) will kill old zk connection for HConnection
and rebuild new zk connection:
2013-05-16 17:04:09,037 INFO  [79168696@qtp-2077588437-0] org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper:
Closing dead ZooKeeper connection, session was: 0x0
2013-05-16 17:04:09,037 DEBUG [79168696@qtp-2077588437-0] org.apache.zookeeper.ZooKeeper:
Closing session: 0x0
2013-05-16 17:04:09,037 DEBUG [79168696@qtp-2077588437-0] org.apache.zookeeper.ClientCnxn:
Closing client for session: 0x0
2013-05-16 17:04:09,131 DEBUG [79168696@qtp-2077588437-0] org.apache.zookeeper.ClientCnxn:
Disconnecting client for session: 0x0
2013-05-16 17:04:09,131 INFO  [79168696@qtp-2077588437-0] org.apache.zookeeper.ZooKeeper:
Session: 0x0 closed
2013-05-16 17:04:09,131 INFO  [79168696@qtp-2077588437-0] org.apache.zookeeper.ZooKeeper:
Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection-0x43724372-0x13eac8abdbc0003-0x13eac917e7e0003-0x13eac917e7e0004
Then, response successfully with table list.

> [REST] HBaseAdmin does not ride over cluster restart
> ----------------------------------------------------
>                 Key: HBASE-8533
>                 URL: https://issues.apache.org/jira/browse/HBASE-8533
>             Project: HBase
>          Issue Type: Improvement
>          Components: REST, scripts
>    Affects Versions: 0.94.3, 0.98.0
>            Reporter: Julian Zhou
>            Priority: Minor
>             Fix For: 0.94.3, 0.98.0
>         Attachments: hbase-8533-trunk-v0.patch
> For Restful servlet (org.apache.hadoop.hbase.rest.Main (0.94), org.apache.hadoop.hbase.rest.RESTServer
(trunk)) on Jetty, we need to first explicitly start the service (% ./bin/hbase-daemon.sh
start rest -p 8000 ) for application running. Here is a scenario, sometimes, HBase cluster
are stopped/started for maintanence, but rest is a seperated standalone process, which binds
the HBaseAdmin at construction method.
> HBase stop/start cause this binding lost for existing rest servlet. Rest servlet still
exist to trying on old bound HBaseAdmin until a long time duration later with an "Unavailable"
caught via an IOException caught in
> such as RootResource.
> Could we pairwise the HBase service with HBase rest service with some start/stop options?
since seems no reason to still keep the rest servlet process after HBase stopped? When HBase
restarts, original rest service could not resume to bind to the new HBase service via its
old HBaseAdmin reference?
> So may we stop the rest when hbase stopped, or even if hbase was killed by acident, restart
hbase with rest option could detect the old rest process, kill it and start to bind a new
> From this point of view, application rely on rest api in previous scenario could immediately
detect it when setting up http connection session instead of wasting a long time to fail back
from IOException with "Unavailable" from rest servlet.
> Put current options from the discussion history here from Andrew, Stack and Jean-Daniel,
> 1) create an HBaseAdmin on demand in rest servlet instead of keeping singleton instance;
(another possible enhancement for HBase client: automatic reconnection of an open HBaseAdmin
handle after a cluster bounce?)
> 2) pairwise the rest webapp with hbase webui so the rest is always on with HBase serive;
> 3) add an option for rest service (such as HBASE_MANAGES_REST) in hbase-env.sh, set HBASE_MANAGES_REST
to true, the scripts will start/stop the REST server.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message