lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5152) Lucene FST is not immutable
Date Thu, 01 Aug 2013 20:33:48 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13726847#comment-13726847
] 

Simon Willnauer commented on LUCENE-5152:
-----------------------------------------

bq. Should we clone payload bytes in the postings lists too? what about term dictionaries?
I agree we can be less conservative here and just use the payload and copy it into a new BytesRef
or whatever is needed. I will bring up a new patch.

bq. At some point then BytesRef is useless as a reference class because of a few bad apples
trying to use it as a ByteBuffer. Ideally we would remove code that abuses BytesRef as a ByteBuffer
instead.

agreed again. We just need to make sure that we have asserts in place that check for that.


bq. I don't mean to pick on your issue Simon, and it doesnt mean I object to the patch (though
I wonder about performance implications), I just see this as one of many in a larger issue.

no worries. I am really concerned about this since it took me forever to figure out the problems
this caused. I just wanna have an infra in place that catches those problems. I am more concerned
about users that get bitten by this. I agree we should figure out the bigger problem eventually
but lets make sure that we fix the bad apples first
                
> Lucene FST is not immutable
> ---------------------------
>
>                 Key: LUCENE-5152
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5152
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/FSTs
>    Affects Versions: 4.4
>            Reporter: Simon Willnauer
>            Priority: Blocker
>             Fix For: 5.0, 4.5
>
>         Attachments: LUCENE-5152.patch, LUCENE-5152.patch
>
>
> a spinnoff from LUCENE-5120 where the analyzing suggester modified a returned output
from and FST (BytesRef) which caused sideffects in later execution. 
> I added an assertion into the FST that checks if a cached root arc is modified and in-fact
this happens for instance in our MemoryPostingsFormat and I bet we find more places. We need
to think about how to make this less trappy since it can cause bugs that are super hard to
find.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message