lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Simon Willnauer (JIRA)" <>
Subject [jira] [Commented] (LUCENE-5152) Lucene FST is not immutable
Date Thu, 01 Aug 2013 20:33:48 GMT


Simon Willnauer commented on LUCENE-5152:

bq. Should we clone payload bytes in the postings lists too? what about term dictionaries?
I agree we can be less conservative here and just use the payload and copy it into a new BytesRef
or whatever is needed. I will bring up a new patch.

bq. At some point then BytesRef is useless as a reference class because of a few bad apples
trying to use it as a ByteBuffer. Ideally we would remove code that abuses BytesRef as a ByteBuffer

agreed again. We just need to make sure that we have asserts in place that check for that.

bq. I don't mean to pick on your issue Simon, and it doesnt mean I object to the patch (though
I wonder about performance implications), I just see this as one of many in a larger issue.

no worries. I am really concerned about this since it took me forever to figure out the problems
this caused. I just wanna have an infra in place that catches those problems. I am more concerned
about users that get bitten by this. I agree we should figure out the bigger problem eventually
but lets make sure that we fix the bad apples first
> Lucene FST is not immutable
> ---------------------------
>                 Key: LUCENE-5152
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/FSTs
>    Affects Versions: 4.4
>            Reporter: Simon Willnauer
>            Priority: Blocker
>             Fix For: 5.0, 4.5
>         Attachments: LUCENE-5152.patch, LUCENE-5152.patch
> a spinnoff from LUCENE-5120 where the analyzing suggester modified a returned output
from and FST (BytesRef) which caused sideffects in later execution. 
> I added an assertion into the FST that checks if a cached root arc is modified and in-fact
this happens for instance in our MemoryPostingsFormat and I bet we find more places. We need
to think about how to make this less trappy since it can cause bugs that are super hard to

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message