lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <j...@apache.org>
Subject [jira] [Created] (LUCENE-5123) invert the codec postings API
Date Fri, 19 Jul 2013 16:48:50 GMT
Robert Muir created LUCENE-5123:
-----------------------------------

             Summary: invert the codec postings API
                 Key: LUCENE-5123
                 URL: https://issues.apache.org/jira/browse/LUCENE-5123
             Project: Lucene - Core
          Issue Type: Wish
            Reporter: Robert Muir


Currently FieldsConsumer/PostingsConsumer/etc is a "push" oriented api, e.g. FreqProxTermsWriter
streams the postings at flush, and the default merge() takes the incoming codec api and filters
out deleted docs and "pushes" via same api (but that can be overridden).

It could be cleaner if we allowed for a "pull" model instead (like DocValues). For example,
maybe FreqProxTermsWriter could expose a Terms of itself and just passed this to the codec
consumer.

This would give the codec more flexibility to e.g. do multiple passes if it wanted to do things
like encode high-frequency terms more efficiently with a bitset-like encoding or other things...

A codec can try to do things like this to some extent today, but its very difficult (look
at buffering in Pulsing). We made this change with DV and it made a lot of interesting optimizations
easy to implement...



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message