Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Tue, 6 Mar 2012 17:44:57 +0000 (UTC)
From: "Benson Margulies (Commented) (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: 
 <538913846.28277.1331055897980.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <303482760.27537.1331045577014.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (LUCENE-3854) Non-tokenized fields become
 tokenized when a document is deleted and added back
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223461#comment-13223461 ] 

Benson Margulies commented on LUCENE-3854:
------------------------------------------

Notes:

1) The trap opened a bit wider in 4.0 with the removal of IndexReader.deleteDocument. I'm not sure I exactly understand how, but by deleting through the reader we didn't hit this.

2) I got into this because I wanted, really, to update a field value. There isn't a better way to attack that problem in 4.0, is there?

                
> Non-tokenized fields become tokenized when a document is deleted and added back
> -------------------------------------------------------------------------------
>
>                 Key: LUCENE-3854
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3854
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 4.0
>            Reporter: Benson Margulies
>
> https://github.com/bimargulies/lucene-4-update-case is a JUnit test case that seems to show a problem with the current trunk. It creates a document with a Field typed as StringField.TYPE_STORED and a value with a "-" in it. A TermQuery can find the value, initially, since the field is not tokenized.
> Then, the case reads the Document back out through a reader. In the copy of the Document that gets read out, the Field now has the tokenized bit turned on. 
> Next, the case deletes and adds the Document. The 'tokenized' bit is respected, so now the field gets tokenized, and the result is that the query on the term with the - in it no longer works.
> So I think that the defect here is in the code that reconstructs the Document when read from the index, and which turns on the tokenized bit.
> I have an ICLA on file so you can take this code from github, but if you prefer I can also attach it here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org