lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (Commented) (JIRA)" <>
Subject [jira] [Commented] (LUCENE-3738) Be consistent about negative vInt/vLong
Date Fri, 16 Mar 2012 18:31:40 GMT


Uwe Schindler commented on LUCENE-3738:

bq. Hmm, I don't think we should do that.

It costs nothing, as the standard vInt will be read only some 1 or 2 bytes, if you really
read until the last byte, you have so big vInts that it might even be better not to use vInts
at all.- And: The not-unrolled loops do the check always.

bq. If you go and edit your index with a hex editor... there are no guarantees on what may

Disk IO can produce wrong data. We must check this if we can and it costs nothing, which is
the case here (see above).

I was already talking with Robert, there are other asserts in the index readiung code at places
completely outside any loops, executed only once when index is opened. Its first priority
to do consistency checks of the read bytes. Otherwise you can even produce endless loops at
some places. - Of course not when you have tight loops, but things like checking that the
document count is in line with e.g. some other value from liveDocs is essential. I will open
an issue for that, the older Lucene formats are much besster secured, but trunk is horrible,
just because some people here seem to want to prevent any check, which is also a security
issue when you e.g. download indexes through network connections and a man in the middle modifies
the stream. 
> Be consistent about negative vInt/vLong
> ---------------------------------------
>                 Key: LUCENE-3738
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Michael McCandless
>             Fix For: 3.6, 4.0
>         Attachments: LUCENE-3738.patch, LUCENE-3738.patch, LUCENE-3738.patch
> Today, write/readVInt "allows" a negative int, in that it will encode and decode correctly,
just horribly inefficiently (5 bytes).
> However, read/writeVLong fails (trips an assert).
> I'd prefer that both vInt/vLong trip an assert if you ever try to write a negative number...
it's badly trappy today.  But, unfortunately, we sometimes rely on this... had we had this
assert in 'since the beginning' we could have avoided that.
> So, if we can't add that assert in today, I think we should at least fix readVLong to
handle negative longs... but then you quietly spend 9 bytes (even more trappy!).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message