Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of
 marcelo.schneider@digitro.com.br designates 189.85.128.24 as permitted
 sender)
Message-ID: <48CA58D6.1070700@digitro.com.br>
Date: Fri, 12 Sep 2008 08:56:06 -0300
From: Marcelo Schneider <marcelo.schneider@digitro.com.br>
User-Agent: Thunderbird 2.0.0.0 (Windows/20070326)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: Spawn an indexing thread on every update
References: <19444244.post@talk.nabble.com>
	 <8c4e68610809120229s7db02303w47504d90f60a7340@mail.gmail.com>
 <f18c9dde0809120406t6d80d1ffia2be808d6f3ee35d@mail.gmail.com>
In-Reply-To: <f18c9dde0809120406t6d80d1ffia2be808d6f3ee35d@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Our application does exactly that: a background thread 
(Thread.MIN_PRIORITY) that buffers every "waiting" document and indexes 
every ~3 minutes, then reopens the searcher. No problems so far...


*
--
*Marcelo Frantz Schneider
SIC - TCO - Tecnologia em Engenharia do Conhecimento
DÍGITRO TECNOLOGIA*

*
Simon Willnauer escreveu:
> I guess similar problems have been discussed on the list over and over again.
> Why don't you use a single update thread which takes care of batch
> processing, commit threshold or performs commits after a certain time
> span.
> This would at least prevent you from spawning all the threads which
> is, as far as I can tell not necessary.
> Your update thread could also notify you searcher to reopen the index
> as soon as a change or a batch was commited.
> If you are looking for examples you might find some good examples in
> the Solr code.
>
> - simon
>
> On Fri, Sep 12, 2008 at 11:29 AM, Ian Lea <ian.lea@gmail.com> wrote:
>   
>> Hi
>>
>>
>> Why 15 mins?  Can you try lower values to get a balance between load
>> and freshness?
>>
>>
>>
>> --
>> Ian.
>>
>>
>> On Thu, Sep 11, 2008 at 9:43 PM, nobody <dailykos@budweiser.com> wrote:
>>     
>>> Hi,
>>>
>>>  In our application, I want users to be able to search for the updates they
>>> make almost immediately. Hence, whenever they update, I spawn a thread
>>> immediately to index. However, when the load on the application is very high
>>> the number of threads spawned increases, and this results in "cannot create
>>> native thread" error.
>>>
>>> We are thinking of running the indexing thread once in every 15 mins,
>>> through a scheduler, and buffer all the writes till the thread runs.
>>> However, this will result in stale results on the search. Please advice what
>>> is the best approach in this regard.
>>>
>>> -thanks
>>> --
>>>       
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>     
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>   

-- 
Esta mensagem foi verificada pelo sistema de antiv�rus da D�gitro e
acredita-se estar livre de perigo.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org