Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Message-ID: <10507182.41201275015279528.JavaMail.jira@thor>
Date: Thu, 27 May 2010 22:54:39 -0400 (EDT)
From: "Hoss Man (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Subject: [jira] Updated: (SOLR-1662) BufferedTokenStream incorrect cloning
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/SOLR-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hoss Man updated SOLR-1662:
---------------------------

    Fix Version/s: 1.5
                   3.1
                   4.0


Correcting Fix Version based on CHANGES.txt, see this thread for more details...

http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E


> BufferedTokenStream incorrect cloning
> -------------------------------------
>
>                 Key: SOLR-1662
>                 URL: https://issues.apache.org/jira/browse/SOLR-1662
>             Project: Solr
>          Issue Type: Bug
>          Components: Schema and Analysis
>    Affects Versions: 1.4
>            Reporter: Robert Muir
>            Assignee: Shalin Shekhar Mangar
>             Fix For: 1.5, 3.1, 4.0
>
>         Attachments: SOLR-1662.patch
>
>
> As part of writing tests for SOLR-1657, I rewrote one of the base classes (BaseTokenTestCase) to use the new TokenStream API, but also with some additional safety.
> {code}
>  public static String tsToString(TokenStream in) throws IOException {
>     StringBuilder out = new StringBuilder();
>     TermAttribute termAtt = (TermAttribute) in.addAttribute(TermAttribute.class);
>     // extra safety to enforce, that the state is not preserved and also
>     // assign bogus values
>     in.clearAttributes();
>     termAtt.setTermBuffer("bogusTerm");
>     while (in.incrementToken()) {
>       if (out.length() > 0)
>         out.append(' ');
>       out.append(termAtt.term());
>       in.clearAttributes();
>       termAtt.setTermBuffer("bogusTerm");
>     }
>     in.close();
>     return out.toString();
>   }
> {code}
> Setting the term text to bogus values helps find bugs in tokenstreams that do not clear or clone properly. In this case there is a problem with a tokenstream AB_AAB_Stream in TestBufferedTokenStream, it converts A B -> A A B but does not clone, so the values get overwritten.
> This can be fixed in two ways: 
> * BufferedTokenStream does the cloning
> * subclasses are responsible for the cloning
> The question is which one should it be?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org