hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2952) HConnectionManager's shutdown hook interferes with client's operations
Date Thu, 02 Sep 2010 00:16:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905333#action_12905333
] 

stack commented on HBASE-2952:
------------------------------

They can call flush on their table to empty out the write buffer but that should be it.

The shutdown hook doesn't seem to be really necessary yet it makes this issue you describe
above and similar for benoit over in 2669.  Let me test this evening and will commit if all
is good w/ TOF.

> HConnectionManager's shutdown hook interferes with client's operations
> ----------------------------------------------------------------------
>
>                 Key: HBASE-2952
>                 URL: https://issues.apache.org/jira/browse/HBASE-2952
>             Project: HBase
>          Issue Type: Improvement
>          Components: client
>    Affects Versions: 0.20.0
>            Reporter: Prakash Khemani
>
> My HBase client calls incrementColValue() in pairs. If someone kills the client (SIGINT
or SIGTERM) I want my client's increment threads to gracefully exit. If a thread has already
done one of the incrementColValue() then I want that thread to complete the other incrementColValue()
and then exit.
> For this purpose I installed my own shutdownHook(). My shitdownHook() thread 'sugnals'
all the threads in my process that it is time to exit and then waits for them to complete.
> The problem is that HConnectionManager's shutdownHook thread also runs and shuts down
all connections and IPC threads.
> My increment thread keeps waiting to increment and then times out after 240s. Two problems
with this - the incrementColValiue() didn't go through which will increase the chances of
inconsistency in my HBase data. And it too 240s to exit. I am pasting some of the messages
that the client thread outputs while it tries contact the HBase server.
> Signalled. Exiting ...
> 2010-09-01 12:11:14,769 DEBUG [HCM.shutdownHook] zookeeper.ZooKeeperWrapper(787): <localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Closed
connection with ZooKeeper; /hbase/root-region-server
> flushing after 7899
> 2010-09-01 12:11:19,669 DEBUG [Line Processing Thread 0] client.HConnectionManager$TableServers(903):
Cache hit for row <> in tableName .META.: location server hadoop2205.snc3.facebook.com:60020,
location region name .META.,,1.1028785192
> 2010-09-01 12:11:19,671 INFO  [Line Processing Thread 0] zookeeper.ZooKeeperWrapper(206):
Reconnecting to zookeeper
> 2010-09-01 12:11:19,671 DEBUG [Line Processing Thread 0] zookeeper.ZooKeeperWrapper(212):
<localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Connected to zookeeper
again
> 2010-09-01 12:11:24,679 DEBUG [Line Processing Thread 0] client.HConnectionManager$TableServers(964):
Removed .META.,,1.1028785192 for tableName=.META. from cache because of content_action_url_metrics,\x080r&
B\xF7\x81_T\x07\x08\x16uOrcom.gigya 429934274290948,99999999999999
> 2010-09-01 12:11:24,680 DEBUG [Line Processing Thread 0] client.HConnectionManager$TableServers(857):
locateRegionInMeta attempt 0 of 4 failed; retrying after sleep of 5000 because: The client
is stopped
> 2010-09-01 12:11:24,680 DEBUG [Line Processing Thread 0] zookeeper.ZooKeeperWrapper(470):
<localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Trying to read /hbase/root-region-server
> 2010-09-01 12:11:24,681 DEBUG [Line Processing Thread 0] zookeeper.ZooKeeperWrapper(489):
<localhost:/hbase,org.apache.hadoop.hbase.client.HConnectionManager>Read ZNode /hbase/root-region-server
got 10.26.119.190:60020
> 2010-09-01 12:11:24,681 DEBUG [Line Processing Thread 0] client.HConnectionManager$TableServers(1116):
Root region location changed. Sleeping.
> ===
> It might be a good idea to only run the HCM shutdown code when all the HTables referring
to it have been closed. That way the client can control when the shutdown actually happens.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message