lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-4658) Per-segment tracking of external/side-car data
Date Sat, 16 Mar 2013 07:48:12 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-4658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13604150#comment-13604150
] 

Shai Erera commented on LUCENE-4658:
------------------------------------

Throwing a wild idea, I'm not sure if it can work. What if we support that by introducing
a CustomField which takes an Object as its value, and IW's chain will forward those per-document
values to Codec.CustomDataFormat? CustomField, I think, should not support FieldType settings,
but hardcode to not stored, inedxed \(?\), not tokenized etc. Codec will support CustomDataFormat
and IndexReader will expose {{Object getCustomData(field)}}. Can this work?
                
> Per-segment tracking of external/side-car data
> ----------------------------------------------
>
>                 Key: LUCENE-4658
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4658
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>         Attachments: LUCENE-4658.patch, LUCENE-4658.patch
>
>
> Spinoff from David's idea on LUCENE-4258
> (https://issues.apache.org/jira/browse/LUCENE-4258?focusedCommentId=13534352&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13534352
)
> I made a prototype patch that allows custom per-segment "side-car
> data".  It adds an abstract ExternalSegmentData class.  The idea is
> the app implements this, and IndexWriter will pass each Document
> through to it, and call on it to do flushing/merging.  I added a
> setter to IndexWriterConfig to enable it, but I think this would
> really belong in Codec ...
> I haven't tackled the read-side yet, though this is already usable
> without that (ie, the app can just open its own files, read them,
> etc.).
> The random test case passes.
> I think for example this might make it easier for Solr/ElasticSearch
> to implement things like ExternalFileField.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message