lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Hatcher <erik.hatc...@gmail.com>
Subject Re: Where to free Tokenizer resources?
Date Tue, 20 Oct 2009 19:36:43 GMT
What about acquiring the resource in your tokenizer factory instead of  
at the tokenizer level?

	Erik


On Oct 20, 2009, at 1:16 PM, Teruhiko Kurosaka wrote:

>
> Yonik,
>
>> If you really want to release/acquire your resources each
>> time the tokenizer is used, then release it in the close()
>> and acquire in the reset().  There is no "done with this
>> forever" callback.
>
> I wanted to avoid that because acquring this resource
> is a relatively expensive operation.  I wanted to do
> that per instance.  I guess I should lobby Lucene folks
> and ask them to consider adding a new method to do so.
>
> Is my guess that Solr calls Tokenizer.close() more than
> once correct? My observation of the behavior suggets
> it but I couldn't find a concrete evidence in the source.
>
>
>>
>> -Yonik
>> http://www.lucidimagination.com
>>
>> On Tue, Oct 20, 2009 at 12:25 PM, Teruhiko Kurosaka
>> <Kuro@basistech.com> wrote:
>>> Hi,
>>> I have my own Tokenizer that was working with Solr 1.3 fine
>> but threw an Exception when used with Solr 1.4 dev.
>>>
>>> This Tokenizer uses some JNI-side resources that it takes
>> in the constructor and it frees it in close().
>>>
>>> The behavior seems to indicate that Solr 1.4 calls close()
>> then reset(Reader) in order to reuse the Tokenizer.  But my
>> Tokenizer threw an Exception because its resource has been
>> freed already. My temporary fix was to move the resource
>> release code from close() to finalize().  But I'm not very
>> happy with it because the timing of resource release is up to
>> the garbage collector.
>>>
>>> Question #1: Is close() supposed to be called more than
>> once? To me,
>>> close() should be called only once at the end of life cycle of the
>>> Tokenizer.  (The old reader shold be closed when reset(Reader) is
>>> called.)
>>>
>>> If the answer is Yes, then
>>>
>>> Question #2: Is there any better place to release the
>> internal resource than in finalize()?
>>>
>>> Thank you.
>>>
>>> T. "Kuro" Kurosaka
>>>
>>>
>>


Mime
View raw message