lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (LUCENE-3848) basetokenstreamtestcase should fail if tokenstream starts with posinc=0
Date Sun, 04 Mar 2012 19:40:59 GMT
basetokenstreamtestcase should fail if tokenstream starts with posinc=0
-----------------------------------------------------------------------

                 Key: LUCENE-3848
                 URL: https://issues.apache.org/jira/browse/LUCENE-3848
             Project: Lucene - Java
          Issue Type: Bug
            Reporter: Robert Muir
             Fix For: 3.6, 4.0


This is meaningless for a tokenstream to start with posinc=0,

Its also caused problems and hairiness in the indexer (LUCENE-1255, LUCENE-1542),
and it makes senseless tokenstreams. We should add a check and fix any that do this.

Furthermore the same bug can exist in removing-filters if they have enablePositionIncrements=false.
I think this option is useful: but it shouldnt mean 'allow broken tokenstream', it just means
we
don't add gaps. 

If you remove tokens with enablePositionIncrements=false it should not cause the TS to start
with
positionincrement=0, and it shouldnt 'restructure' the tokenstream (e.g. moving synonyms on
top of a different word).
It should just not add any 'holes'.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message