hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Héctor Izquierdo (JIRA) <j...@apache.org>
Subject [jira] Commented: (HBASE-2669) HCM.shutdownHook causes data loss with hbase.client.write.buffer != 0
Date Wed, 28 Jul 2010 15:15:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893201#action_12893201

Héctor Izquierdo commented on HBASE-2669:

An ugly workaround:

private Thread getEvilHTableShutdownHook() throws Exception {

        Class clazz = Class.forName("java.lang.ApplicationShutdownHooks");

        Field[] fields = clazz.getDeclaredFields();

        for (Field field : fields) {

            if (field.getType() == IdentityHashMap.class) {
                IdentityHashMap<Thread, Thread> hooks = (IdentityHashMap<Thread,
Thread>) field.get(null);

                for (Map.Entry<Thread, Thread> entries : hooks.entrySet()) {
                    if (entries.getValue().getName().equals("HCM.shutdownHook")) {
                        return entries.getValue();


        return null;

> HCM.shutdownHook causes data loss with hbase.client.write.buffer != 0
> ---------------------------------------------------------------------
>                 Key: HBASE-2669
>                 URL: https://issues.apache.org/jira/browse/HBASE-2669
>             Project: HBase
>          Issue Type: Bug
>          Components: client
>            Reporter: Benoit Sigoure
>            Assignee: Benoit Sigoure
>            Priority: Critical
>             Fix For: 0.90.0
> In my application I set {{hbase.client.write.buffer}} to a reasonably small value (roughly
64 edits) in order to try to batch a few {{Put}} together before talking to HBase.  When my
application does a graceful shutdown, I call {{HTable#flushCommits}} in order to flush any
pending change to HBase.  I want to do the same thing when I get a {{SIGTERM}} by using {{Runtime#addShutdownHook}}
but this is impossible since {{HConnectionManager}} already registers a shutdown hook that
invokes {{HConnectionManager#deleteAllConnections}}.  This static method closes all the connections
to HBase and then all connections to ZooKeeper.  Because all shutdown hooks run in parallel,
my hook will attempt to flush edits while connections are getting closed.
> There is no way to guarantee the order in which the hooks will execute, so I propose
that we remove the hook in the HCM altogether and provide some user-visible API they call
in their own hook after they're done flushing their stuff, if they really want to do a graceful
shutdown.  I expect that a lot of users won't use a hook though, otherwise this issue would
have cropped up already.  For those users, connections won't get "gracefully" terminated,
but I don't think that would be a problem since the underlying TCP socket will get closed
by the OS anyway, so things like ZooKeeper and such should realize that the connection has
been terminated and assume the client is gone, and do the necessary clean-up on their side.
> An alternate fix would be to leave the hook in place by default but keep a reference
to it and add a user-visible API to be able to un-register the hook.  I find this ugly.
> Thoughts?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message