lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manish Shukla <hardtopm...@yahoo.com>
Subject Re: Modifying document with unstored fields
Date Mon, 26 Aug 2002 04:03:43 GMT
We had the same problem, we wanted to add new tokens
to a field ( reagrdless of it being stored on unstored
) We also wanted to explore the possiblity of adding
the adding new fields to a document. 

Deleting and re-adding document didn't seem to scale
because it involves re-tokenizing documents and big
document use a lot of cpu of tokenize.  

One solution, which I have been trying to work on,
involves reading ".tis" and ".tii" files and
resurrecting the unstored tokens.

Do any of you have any insights into into how to go
about it or any other pointers.

-Manish


--- Dmitry Serebrennikov <dmitrys@earthlink.net>
wrote:
> Victor Hadianto wrote:
> 
> >On Thu, 22 Aug 2002 23:14, Otis Gospodnetic wrote:
> >  
> >
> >>That's the very top question/answer in Lucene FAQ
> at jGuru:
> >>http://www.jguru.com/faq/Lucene
> >>    
> >>
> >
> >Hi Otis,
> >
> >Yep I realise that, but I think you haven't read my
> question closely. My 
> >problem is not simply delete/add the new document,
> but what happen with the 
> >fields thare are unstored. If all the fields in my
> documents are stored then 
> >it should be fine, but unfortunately not in our
> current situation.
> >
> >Has anyone else ever came across this problem?
> >
> Yes, generally, there are two answers -- either make
> all fields stored 
> or use some other database for the storage of the
> "master" documents. 
> The first approach expands the use of Lucene to the
> point where it 
> becomes a database rather then an indexing engine.
> This makes sense in 
> some applications, but generally degrades in
> performance. The second 
> approach is preferred, but it increases complexity
> of the overall solution.
> 
> I suppose it might be possible to add a lower-level
> support for 
> extracting the terms from the unstored fields of the
> initial document 
> and re-indexing them for the new document...
> 
> Dmitry.
> 
> >
> >  
> >
> >>Otis
> >>    
> >>
> >
> >Victor
> >
> >
> >  
> >
> >>--- Victor Hadianto <victorh@nuix.com.au> wrote:
> >>    
> >>
> >>>Hi,
> >>>
> >>>I asked this question yesterday on the user-list
> and so far there is
> >>>no
> >>>reply. I post again this question on the dev-list
> hoping that someone
> >>>can
> >>>answer it here.
> >>>
> >>>We have a situation where we have a large
> collection of documents,
> >>>which
> >>>consist of both stored and unstored fields, and
> we'd like to
> >>>add/modify a
> >>>stored field on an existing document.
> >>>
> >>>It seems the only way this can be achieved is to
> delete the document,
> >>>and
> >>>then re-create it. However, this will only
> perserve stored fields,
> >>>the
> >>>unstored field information will be lost.
> >>>
> >>>In our application, the unstored fields consist
> of very large data,
> >>>and it
> >>>would not be desirable to store them.
> >>>
> >>>Are there any ways in getting around this
> problem?  Thanks.
> >>>
> >>>--
> >>>Victor Hadianto
> >>>
> >>>NUIX Pty Ltd
> >>>Level 8, 143 York Street, Sydney 2000
> >>>Phone: (02) 9283 9010
> >>>Fax:   (02) 9283 9020
> >>>
> >>>This message is intended only for the named
> recipient. If you are not
> >>>the
> >>>intended recipient you are notified that
> disclosing, copying,
> >>>distributing
> >>>or taking any action in reliance on the contents
> of this message or
> >>>attachment is strictly prohibited.
> >>>
> >>>--
> >>>To unsubscribe, e-mail:
>
>>><mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> >>>For additional commands, e-mail:
> >>><mailto:lucene-dev-help@jakarta.apache.org>
> >>>      
> >>>
> >>__________________________________________________
> >>Do You Yahoo!?
> >>HotJobs - Search Thousands of New Jobs
> >>http://www.hotjobs.com
> >>    
> >>
> >
> >  
> >
> 
> 
> 
> 
> --
> To unsubscribe, e-mail:  
> <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail:
> <mailto:lucene-dev-help@jakarta.apache.org>
> 


__________________________________________________
Do You Yahoo!?
Yahoo! Finance - Get real-time stock quotes
http://finance.yahoo.com

--
To unsubscribe, e-mail:   <mailto:lucene-dev-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-dev-help@jakarta.apache.org>


Mime
View raw message