lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shay Banon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2960) Allow (or bring back) the ability to setRAMBufferSizeMB on an open IndexWriter
Date Sat, 12 Mar 2011 14:55:59 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13006049#comment-13006049
] 

Shay Banon commented on LUCENE-2960:
------------------------------------

Heya,

   If I had to choose between being able to change things in real time to better concurrency
thanks to immutability, I would definitely go with better concurrency. I have no problems
with closing the writers and reopening them, though, as Mike said, this can come with a big
cost.

   The funny thing is that a lot of the setters that were already there on the IndexWriter
are still exposed, basically, through settings on the relevant MergePolicy, so I don't think
we are talking about that many setter to begin with (I don't think we should bring those back
to the IndexWriter).

   I think that the notion of IWC is a good one, and should remain, but only to provide construction
time parameters to IW. It should not be consulted once the construction phase of IW is done.
If explicit real time parameters are to be set, then IW should expose it as a setter. Now,
the question is which, if any, setters should be exposed.

   Going through the list of current setters on IW, my vote is for the setRAMBufferSizeMB
one. I am not sure that its that obscure use case. I believe Solr for example has a notion
of cores (or something like that), so it can also be adaptive in terms of indexing buffer
size dependent on the number of cores running in the VM. Also, one can easily run a system
where it does bulk indexing, and then lowers the indexing buffer size for more "streamline"
work. Its just a shame to close the writer for that (and having to pause all indexing work
while this happens).

   The term interval and divisor, I agree, are such obscure (funnily, I use the divisor quite
a lot), that closing the writer and opening it again make sense.

> Allow (or bring back) the ability to setRAMBufferSizeMB on an open IndexWriter
> ------------------------------------------------------------------------------
>
>                 Key: LUCENE-2960
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2960
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>            Reporter: Shay Banon
>            Priority: Blocker
>             Fix For: 3.1, 4.0
>
>
> In 3.1 the ability to setRAMBufferSizeMB is deprecated, and removed in trunk. It would
be great to be able to control that on a live IndexWriter. Other possible two methods that
would be great to bring back are setTermIndexInterval and setReaderTermsIndexDivisor. Most
of the other setters can actually be set on the MergePolicy itself, so no need for setters
for those (I think).

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message