lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (Commented) (JIRA)" <>
Subject [jira] [Commented] (LUCENE-3848) basetokenstreamtestcase should fail if tokenstream starts with posinc=0
Date Thu, 15 Mar 2012 20:22:38 GMT


Robert Muir commented on LUCENE-3848:

I think this is ready to go in, ill wait a bit.

I didn't make any changes re: "graph-restructuring", though I still feel we should fix this,
but it means
dealing with backwards compatibility, etc.

The changes in this patch are backwards compatible, in the sense that consumers are already
'initial posInc=0' to posinc=1 anyway.

> basetokenstreamtestcase should fail if tokenstream starts with posinc=0
> -----------------------------------------------------------------------
>                 Key: LUCENE-3848
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Robert Muir
>             Fix For: 4.0
>         Attachments: LUCENE-3848-MockGraphTokenFilter.patch, LUCENE-3848.patch, LUCENE-3848.patch
> This is meaningless for a tokenstream to start with posinc=0,
> Its also caused problems and hairiness in the indexer (LUCENE-1255, LUCENE-1542),
> and it makes senseless tokenstreams. We should add a check and fix any that do this.
> Furthermore the same bug can exist in removing-filters if they have enablePositionIncrements=false.
> I think this option is useful: but it shouldnt mean 'allow broken tokenstream', it just
means we
> don't add gaps. 
> If you remove tokens with enablePositionIncrements=false it should not cause the TS to
start with
> positionincrement=0, and it shouldnt 'restructure' the tokenstream (e.g. moving synonyms
on top of a different word).
> It should just not add any 'holes'.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message