Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Tue, 12 Jun 2012 08:25:44 +0000 (UTC)
From: =?utf-8?Q?Andr=C3=A1s_P=C3=A9teri_=28JIRA=29?= <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: <323666316.6498.1339489544471.JavaMail.jiratomcat@issues-vm>
In-Reply-To: 
 <303482760.27537.1331045577014.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (LUCENE-3854) Non-tokenized fields become
 tokenized when a document is deleted and added back
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/LUCENE-3854?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D132=
93446#comment-13293446 ]=20

Andr=C3=A1s P=C3=A9teri commented on LUCENE-3854:
---------------------------------------

Isn't this considered a regression from 3.x? In 3.6.0 I'm seeing an additio=
nal byte being read from the stream in FieldsReader, which contained bits t=
hat allowed the reader to reconstruct the Index enum correctly for the fiel=
d. This should make it possible to properly update a document in which all =
fields were stored, with the exception of boost values (and they could be s=
tored redundantly in a field as well to overcome this limitation).
               =20
> Non-tokenized fields become tokenized when a document is deleted and adde=
d back
> -------------------------------------------------------------------------=
------
>
>                 Key: LUCENE-3854
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3854
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 4.0
>            Reporter: Benson Margulies
>
> https://github.com/bimargulies/lucene-4-update-case is a JUnit test case =
that seems to show a problem with the current trunk. It creates a document =
with a Field typed as StringField.TYPE_STORED and a value with a "-" in it.=
 A TermQuery can find the value, initially, since the field is not tokenize=
d.
> Then, the case reads the Document back out through a reader. In the copy =
of the Document that gets read out, the Field now has the tokenized bit tur=
ned on.=20
> Next, the case deletes and adds the Document. The 'tokenized' bit is resp=
ected, so now the field gets tokenized, and the result is that the query on=
 the term with the - in it no longer works.
> So I think that the defect here is in the code that reconstructs the Docu=
ment when read from the index, and which turns on the tokenized bit.
> I have an ICLA on file so you can take this code from github, but if you =
prefer I can also attach it here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs: https://issues.apache.org/jira/secure/ContactAdministrators!default.jsp=
a
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org