hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: HTable and threads
Date Tue, 29 Mar 2011 22:56:33 GMT
Are you suggesting that the thread pool be shutdown in this method ?
  public void close() throws IOException {
    flushCommits();
  }


On Tue, Mar 29, 2011 at 3:49 PM, Joe Pallas <pallas@cs.stanford.edu> wrote:

> Trying to understand why out test program was generating so many threads
> (HBase 0.90.0), I discover that every time we instantiate HTable we get a
> new thread pool (ThreadPoolExecutor).  This seems a bit odd.  HTable is not
> thread safe, so every thread needs to have its own HTable, but every HTable
> creates its own thread pool, and the threads are not shared by the different
> HTables.  (Isn't that what would make sense if the HTables are on the same
> table?)
>
> Is this the way things are supposed to work?  What should I be doing to
> avoid creating lots of threads, which seem to hang around for a long time
> even though the HTables are discarded.  (I'm seeing client GC happen, but
> threads don't seem to decrease.) There are some ThreadPoolExecutor
> parameters that seem like they could be relevant.
>
> I found HBASE-3553, which seems related, but only in so far as the fix
> looks like it might make this problem worse.
>
> So: what should a multithreaded client do to be safe and not generate more
> threads than are needed?
>
> Thanks.
> joe
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message