hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Buttler, David" <buttl...@llnl.gov>
Subject RE: HTable and threads
Date Wed, 30 Mar 2011 00:15:03 GMT
Do the static methods on HTable (like isTableEnabled), also have this problem? From the code
it looks like if you naively call the static method without a Configuration object it will
create a configuration and put it into a HashMap where it will live around forever.

This really bit me recently.  Using the HTablePool doesn't really solve this because for some
reason there is no meta operations on the HTableInterface object itself -- I can't ask if
it is enabled.  Is there any particular reason that this method only lives on an HConnectionManager?

Dave


-----Original Message-----
From: Ted Yu [mailto:yuzhihong@gmail.com] 
Sent: Tuesday, March 29, 2011 4:00 PM
To: user@hbase.apache.org
Cc: Jean-Daniel Cryans
Subject: Re: HTable and threads

See https://issues.apache.org/jira/browse/HBASE-3712

On Tue, Mar 29, 2011 at 3:58 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> Yeah after flushing the remaining edits.
>
> On Tue, Mar 29, 2011 at 3:56 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > Are you suggesting that the thread pool be shutdown in this method ?
> >  public void close() throws IOException {
> >    flushCommits();
> >  }
> >
> >
> > On Tue, Mar 29, 2011 at 3:49 PM, Joe Pallas <pallas@cs.stanford.edu>
> wrote:
> >
> >> Trying to understand why out test program was generating so many threads
> >> (HBase 0.90.0), I discover that every time we instantiate HTable we get
> a
> >> new thread pool (ThreadPoolExecutor).  This seems a bit odd.  HTable is
> not
> >> thread safe, so every thread needs to have its own HTable, but every
> HTable
> >> creates its own thread pool, and the threads are not shared by the
> different
> >> HTables.  (Isn't that what would make sense if the HTables are on the
> same
> >> table?)
> >>
> >> Is this the way things are supposed to work?  What should I be doing to
> >> avoid creating lots of threads, which seem to hang around for a long
> time
> >> even though the HTables are discarded.  (I'm seeing client GC happen,
> but
> >> threads don't seem to decrease.) There are some ThreadPoolExecutor
> >> parameters that seem like they could be relevant.
> >>
> >> I found HBASE-3553, which seems related, but only in so far as the fix
> >> looks like it might make this problem worse.
> >>
> >> So: what should a multithreaded client do to be safe and not generate
> more
> >> threads than are needed?
> >>
> >> Thanks.
> >> joe
> >>
> >>
> >
>

Mime
View raw message