lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Re: IndexWriter#setRAMBufferSizeMB removed in trunk
Date Thu, 10 Mar 2011 13:23:19 GMT
IWC simplified IW creation - now there is only one ctor, where before there
were multiple ones, and some settings could only be changed after IW was
created.

With IWC, our code is (can become) simpler -- e.g. RAM buffer size, if
specified up front is one thing, but if it's dynamic, we need to have code
which dynamically increases or decreases it. Increasing is not the problem,
but decreasing requires special code that flushes and discards the extra
memory. Maybe the code already exists, I haven't checked.

I don't like setters that are all over the place either. Having said that
though, today the setters are inconsistent -- some are 'static' (meaning,
cannot change after IW created) while some are dynamic, such as the
MergePolicy settings. Because MP responds to those setters.

One thing we can do is keep all the setters in IWC and have IW pass itself
to IWC after creation. Then, we can modify certain settings in IWC to notify
IW of these changes. But it's complicated.

Another thing is separate some runtime settings from IWC and include them in
IW, like we do for MP ... that's what's been suggested. But then, what is a
'runtime' setting? Someone can decide to have IndexDeletionPolicy change
'on-the-fly' in his app -- does it make sense that we make IDP a runtime
setting? I don't think so.

In fact, I don't think RAM buffer is changed that dynamically by
applications (or any other setter). Elastic search may have a use case where
it's needed, that's ok. If this setting does not change very often, it can
still close IW and reopen it with the new settings, right?

A third solution is to keep IWC for construction time, but introduce the
setters back on IW for runtime changes.That way we keep IW ctor simple but
still allow apps to change on-the-fly settings. We'll dup setters which I
don't like ...

Shai

On Thu, Mar 10, 2011 at 2:47 PM, Robert Muir <rcmuir@gmail.com> wrote:

> On Thu, Mar 10, 2011 at 7:41 AM, Shay Banon <kimchy@gmail.com> wrote:
> > I am not sure that IndexWriterConfig is bad. Its nice to be able to set
> all
> > the upfront configurations in a single object and pass it to the
> > IndexWriter. And, have the IndexWriter allow for specific setters
> allowing
> > for real time changes (those should not be done through the
> > IndexWriterConfig).
> > The question is which real time changes are allowed or not. The fact that
> > they are separated (IndexWriterConfig, and real time setters) is good
> since
> > it allows to distinguish between what can be set when setting up an
> > IndexWriter, compared to what can be set in real time. We did not have
> this
> > distinction before the IndexWriterConfig was introduced.
> > This open the door for optimizations for things that can only be set when
> > constructing an IndexWriter. Usually, supporting real time changes can
> > hinder concurrency, while having parameters that are basically immutable
> > allows to optimize in this case.
> > -shay.banon
> >
>
> I disagree that its good if things are separate... Instead of API
> confusion I think I would prefer a single method on IW that "best
> effort" tries to apply any "realtime" setters
>
> This way we can avoid constant deprecation and undeprecation between
> these APIs. Instead, whether something can be changed on the fly is
> only a documentation issue.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

Mime
View raw message