lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-4658) Per-segment tracking of external/side-car data
Date Sat, 16 Mar 2013 21:42:12 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604430#comment-13604430
] 

Shai Erera commented on LUCENE-4658:
------------------------------------

Hmm ... I thought the idea is to allow the app to pass some data that is not a 'field' per
se. I mean, if I wanted to take advantage of some data that's in the stored fields, I could
make a Codec which wraps that particular format with whatever logic I want to do "on the side"?

I think, but you are definitely the expert here, that if we go the CustomField approach we
allow the app to be as flexible as it needs, without tapping into IndexingChain. It can stuff
into that field whatever combination of fields it added to the document already. That field,
together with CustomDataFormat, gives the app all the freedom that it needs. We already force
the app to add the same field twice if e.g. it wants to add a DV field and store it. I don't
think this case should be treated differently.
                
> Per-segment tracking of external/side-car data
> ----------------------------------------------
>
>                 Key: LUCENE-4658
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4658
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>         Attachments: LUCENE-4658.patch, LUCENE-4658.patch
>
>
> Spinoff from David's idea on LUCENE-4258
> (https://issues.apache.org/jira/browse/LUCENE-4258?focusedCommentId=13534352&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13534352
)
> I made a prototype patch that allows custom per-segment "side-car
> data".  It adds an abstract ExternalSegmentData class.  The idea is
> the app implements this, and IndexWriter will pass each Document
> through to it, and call on it to do flushing/merging.  I added a
> setter to IndexWriterConfig to enable it, but I think this would
> really belong in Codec ...
> I haven't tackled the read-side yet, though this is already usable
> without that (ie, the app can just open its own files, read them,
> etc.).
> The random test case passes.
> I think for example this might make it easier for Solr/ElasticSearch
> to implement things like ExternalFileField.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message