accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: ScannerIterator thread use
Date Tue, 01 Nov 2011 20:31:35 GMT
On Tue, Nov 1, 2011 at 4:00 PM, Keith Massey
<keith.massey@digitalreasoning.com> wrote:
> We're querying accumulo through a web application. After it had been hit
> with one of our test scripts for a few minutes with the debugger attached I
> noticed that there were hundreds and hundreds of threads being garbage
> collected. Eventually it crashes my IDE and the server becomes unresponsive.
> The server recovers eventually. After looking through the code a little bit,
> it appears that these threads are coming from
> org.apache.accumulo.core.client.impl.ScannerIterator.initiateReadAhead(). We
> actually get many threads per iterator. Is there any reason that it can't
> use a thread pool instead of creating a new thread for every call to that
> method?
> Thanks.
>
> Keith
>

The reason I did not use a thread pool is because the scanner does not
have a close method.  I suppose we could use a thread pool where the
threads timeout when not used.  This could still lead to a lot of
threads depending on the timeout and how many scanner iterators are
created.

The BatchScanner and BatchWriter interfaces use thread pools and have
close methods.

Do you think this issue needs a ticket?

Mime
View raw message