lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Earwin Burrfoot (JIRA)" <>
Subject [jira] Commented: (LUCENE-2960) Allow (or bring back) the ability to setRAMBufferSizeMB on an open IndexWriter
Date Sun, 13 Mar 2011 17:46:59 GMT


Earwin Burrfoot commented on LUCENE-2960:

Why such purity? What do we gain?

I'm all for purity, but only if it doesn't interfere w/ functionality.
Here, it's taking away freedom...
We gain consistency and predictability. And there are a lot of freedoms dangerous for developers.

In fact it should be fine to share an IWC across multiple writers; you
can change the RAM buffer for all of them at once.

You've brought up a purrfect example of how NOT to do things.
This is called 'action at a distance' and is a damn bug. Very annoying one.
I've thoroughly experienced it with previous major version of Apache HTTPClient - they had
an API that suggested you can set per-request timeouts, while these were actually global for
a single Client instance.
I fried my brain trying to understand why the hell random user requests timeout at hundred
times their intended duration.
Oh! It was an occasional admin request changing the global.

<irony> You know, you can actually instantiate some DateRangeFilter with a couple of
Dates, and then change these Dates (they are writeable) before each request. Isn't it an exciting
kind of programming freedom?
Or, back to our current discussion - we can pass RAMBufferSizeMB as an AtomicDouble, instead
of current double, then we can use .set() on an instance we passed, and have our live reconfigurability.
What's more - AtomicDouble protects us from word tearing! </irony>

bq. I doubt there's any JVM out there where our lack-of-volatile infoStream causes any problems.
Er.. While I have never personally witnessed unsynchronized long/double tearing,
I've seen the consequence of unsafely publishing a HashMap - an endless loop on get().
It happened on your run off the mill Sun 1.6 JVM.
So the bug is there, lying in wait. Maybe nobody ever actually used the freedom to change
infoStream in-flight, or the guy was lucky, or in his particular situation the field was guarded
by some unrelated sync.

While I see banishing live reconfiguration from IW as a lost cause, I ask to make IWC immutable
at the very least. As Shay said - this will provide a clear barrier between mutable and immutable

> Allow (or bring back) the ability to setRAMBufferSizeMB on an open IndexWriter
> ------------------------------------------------------------------------------
>                 Key: LUCENE-2960
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Shay Banon
>            Priority: Blocker
>             Fix For: 3.1, 4.0
> In 3.1 the ability to setRAMBufferSizeMB is deprecated, and removed in trunk. It would
be great to be able to control that on a live IndexWriter. Other possible two methods that
would be great to bring back are setTermIndexInterval and setReaderTermsIndexDivisor. Most
of the other setters can actually be set on the MergePolicy itself, so no need for setters
for those (I think).

This message is automatically generated by JIRA.
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message