hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Liochon <nkey...@gmail.com>
Subject Re: Connection reference counting
Date Wed, 17 Jul 2013 16:31:22 GMT
I had a look, I think it's a bug.
If the connection holds resources that should be freed, well they won't be
It's used in the minicluster shutdown. So we could imagine scenarios which
that create some test flakyness. I doubt it happens in real life, but imho
it's better to fix it (it there are no different opinions, I will do it if
you don't ;-).



On Wed, Jul 17, 2013 at 10:03 AM, Lars George <lars.george@gmail.com> wrote:

> Hi,
> I am working on an issue around Thrift 2 (HBASE-7035), and I am testing
> how this all works a bit. See
> https://github.com/larsgeorge/hbase-scanner-test for the code, which runs
> against a local HBase instance. Now, there is an issue that connections do
> not get closed as I would have expected. I close the connection in a loop
> over a scanner that gets single rows. Although I close it, it keeps on
> going. Scanners may or may not be reference counted - any code calling
> HCM.getConnection(conf) increases the count.
> So I tried a HCM.deleteAllConnections, which does this:
>   public static void deleteAllConnections() {
>     synchronized (HBASE_INSTANCES) {
>       Set<HConnectionKey> connectionKeys = new HashSet<HConnectionKey>();
>       connectionKeys.addAll(HBASE_INSTANCES.keySet());
>       for (HConnectionKey connectionKey : connectionKeys) {
>         deleteConnection(connectionKey, false);
>       }
>       HBASE_INSTANCES.clear();
>     }
>   }
> It iterates over all connections and "deletes" them. Then is clears the
> entire reference list! The issue is that deleteConnection() is using the
> refcounts and is *not* closing the connection when it is still referenced.
> The final clear() call simply drops them from the list of managed
> connections. This means that we now might have dangling open connections,
> and unless you hold a reference there is no way that you can talk to it
> again, not even using deleteStaleConnection() for example.
> Is that wanted? Just curious, since obviously this is not a biggie in real
> life.
> Lars

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message