lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rutherglen <>
Subject Re: Improving TimeLimitedCollector
Date Wed, 24 Jun 2009 22:35:47 GMT
This would be good however how would we obtain the thread? I
believe this would require using a ThreadLocalish type of system
which could be quite slow (to obtain the thread and lookup in
the hashmap).

One implementation I looked at before was to add this check in
IndexReader.isDeleted (by overriding inside FilterIndexReader).

Perhaps SegmentTermDocs is the best place for this, it could
check a timeout boolean variable which can (somehow) be
optional, meaning, the code doesn't always check this and may
also use a different call path. The boolean variable wouldn't be
volatile as I believe based on our previous discussions is
slower to check than a regular variable. Hopefully this would be
good enough and more accurate than our current implementation.

On Wed, Jun 24, 2009 at 3:07 PM, Mark Harwood <>wrote:

> I think the Collector approach makes the most sense to me, since it's the
>> only object I fully control in the search process. I cannot control Query
>> implementations, and I cannot control the decisions made by IndexSearcher.
>> But I can always wrap someone else's Collector with TLC and pass it to
>> search().
> Isn't another option to have a TimeLimitedIndexReader?
> Clients would need to make a call before the start of any timeout-able
> activity e.g.
>    timeLimitedReader.startActivity(MAX_ALLOWED_DURATION);  //sets a
> ThreadLocal
>   //run query etc
> All invocations on special IndexReader, TermEnum, TermDocs etc subclasses
> can then periodically check to see if the calling thread has exceeded its
> allotted time. This would be a potentially non-invasive way of making all
> filters', queries', scorers' etc read activity subject to some time limit as
> they all typically have to invoke IndexReader-related classes relatively
> frequently as part of their processing.
> Not sure how quick you can make all the underlying time checks though.
> Cheers,
> Mark
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message