lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Teruhiko Kurosaka <K...@basistech.com>
Subject RE: Where to free Tokenizer resources?
Date Tue, 20 Oct 2009 20:34:47 GMT
Erik,
That's a good idea.  

But that means the resource releasing code must live in
the finialize method and it has to wait until GC
kicks in.  Correct?
-kuro  

> -----Original Message-----
> From: Erik Hatcher [mailto:erik.hatcher@gmail.com] 
> Sent: Tuesday, October 20, 2009 12:37 PM
> To: solr-dev@lucene.apache.org
> Subject: Re: Where to free Tokenizer resources?
> 
> What about acquiring the resource in your tokenizer factory 
> instead of at the tokenizer level?
> 
> 	Erik
> 
> 
> On Oct 20, 2009, at 1:16 PM, Teruhiko Kurosaka wrote:
> 
> >
> > Yonik,
> >
> >> If you really want to release/acquire your resources each time the 
> >> tokenizer is used, then release it in the close() and 
> acquire in the 
> >> reset().  There is no "done with this forever" callback.
> >
> > I wanted to avoid that because acquring this resource is a 
> relatively 
> > expensive operation.  I wanted to do that per instance.  I guess I 
> > should lobby Lucene folks and ask them to consider adding a 
> new method 
> > to do so.
> >
> > Is my guess that Solr calls Tokenizer.close() more than 
> once correct? 
> > My observation of the behavior suggets it but I couldn't find a 
> > concrete evidence in the source.
> >
> >
> >>
> >> -Yonik
> >> http://www.lucidimagination.com
> >>
> >> On Tue, Oct 20, 2009 at 12:25 PM, Teruhiko Kurosaka 
> >> <Kuro@basistech.com> wrote:
> >>> Hi,
> >>> I have my own Tokenizer that was working with Solr 1.3 fine
> >> but threw an Exception when used with Solr 1.4 dev.
> >>>
> >>> This Tokenizer uses some JNI-side resources that it takes
> >> in the constructor and it frees it in close().
> >>>
> >>> The behavior seems to indicate that Solr 1.4 calls close()
> >> then reset(Reader) in order to reuse the Tokenizer.  But 
> my Tokenizer 
> >> threw an Exception because its resource has been freed already. My 
> >> temporary fix was to move the resource release code from 
> close() to 
> >> finalize().  But I'm not very happy with it because the timing of 
> >> resource release is up to the garbage collector.
> >>>
> >>> Question #1: Is close() supposed to be called more than
> >> once? To me,
> >>> close() should be called only once at the end of life 
> cycle of the 
> >>> Tokenizer.  (The old reader shold be closed when reset(Reader) is
> >>> called.)
> >>>
> >>> If the answer is Yes, then
> >>>
> >>> Question #2: Is there any better place to release the
> >> internal resource than in finalize()?
> >>>
> >>> Thank you.
> >>>
> >>> T. "Kuro" Kurosaka
> >>>
> >>>
> >>
> 
> 
Mime
View raw message