lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (Commented) (JIRA)" <>
Subject [jira] [Commented] (SOLR-3141) Deprecate OPTIMIZE command in Solr
Date Tue, 28 Feb 2012 01:07:50 GMT


Hoss Man commented on SOLR-3141:

I don't have the energy to really get in depth with all of the discussion that's taken place
so far, i'll try to keep my comments brief:

0) i'm a fan of the patch currently attached.

1) i largely agree with most of yonik's points -- this is a documentation problem first and
foremost.  Saying that all people who optimize are wrong is ridiculous, and breaking something
that has use and value for a set of people just because some other set of people are using
it foolishly seems really absurd.

2) changing the "optimize" command to be a no-op with a warning logged, or a failure, where
the documented "fix" to regain old behavior for people who genuinely need it is to search
& replace the string "optimize" with some new string "forceMerge" seems uterly absurd
to me.  this is not the first time we've had a param name that people later regretted giving
the name that we did -- are we going to change _all_ of them for 4.0?  Unlike a method renamed
in java code where it's easy to see how the change affects you because of compilation failures,
this kind of HTTP param change is a serious pain in the ass for people with client apps written
using multiple languages/libraries ... naming consistency for existing users seems far more
important then having perfect names.

3) Even if the goal is to force people to evaluate whether they really want to merge down
to one segment, we have to consider how hard we make things for people when the answer is
"yes".  If someone is using a client library/app to talk to Solr it may not be easy/simple/possible
for them to replace "optimize" with "forceMerge" or something like it w/o mucking in the internals
of that library -- there's no reason to piss off users like that.

4) any discussion about renaming/removing "optimize" in the Solr HTTP APIs should really consider
how that will impact a few other user visible things...

* {{<listener event="postOptimize" />}} hooks in solrconfig and the corisponding SolrEventListener.postOpimize
* SolrDeletionPolicy has options related to how many optimized indexes to keep
* spellchecker has options relating to building on optimize (although if i remember correctly
there is a bug about this being broken so it can probably die no problem)

5) Assuming that too many people optimize when the shouldn't, either out of ignorance or because
their tools do it out of ignorance and we want to help minimize that moving forward; and given
my opinion that renaming "optimize" will only hurt people w/o actually helping the root problem
-- here's my straw man proposal to try and improve the situation (similar to what jan suggested
but taking into account that we already support a "maxSegments" option when doing optimize
commands) ...

* commit the attached patch as is (it's just plain a good idea, regardless of anything else
we might do)
* change CommitUpdateCommand.maxOptimizeSegments so it defaults to "-1" and document that
when the value is less then 0 it means the UpdateHandler configuration determines the value.
* add a new {{<defaultOptimizeSegments/>}} config option to {{<updateHandler/>}}
- make the UpdateHandler use that value anytime CommitUpdateCommand.maxOptimizeSegments is
less then 0, and for backcompat have it default to "1" if not specified.
* update the example configs to include {{<defaultOptimizeSegments>9999999</defaultOptimizeSegments>}}
with a comment warning against hte evils of over-optimization
* change the code in Solr which deals with {{<optimize ... />}} formated instructions
so that any SolrParams in the request with names the same as xml attributes override the attributes
-- ie: {{POST /update?maxSegments=4 with data: <optimize maxSegments="9" />}} should
result in a CommitUpdateCommand with maxOptimizeSegments=4

The end result being:
* new users who start with new configs have an UpdateHandler that is going effectively ignore
"optimize" commands that don't specify a "maxSegments"
* nothing breaks for existing users
* existing users who only want to allow optimize commands when "maxSegments" is specified
can cut/paste that oneline {{<defaultOptimizeSegments/>}} config
* new and existing users who want Solr to ignore all optimize commands, even when they do
have a "maxSegments", can configure an invariant maxSegments=9999999 param on the affected
request handlers

> Deprecate OPTIMIZE command in Solr
> ----------------------------------
>                 Key: SOLR-3141
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>    Affects Versions: 3.5
>            Reporter: Jan H√łydahl
>              Labels: force, optimize
>             Fix For: 3.6
>         Attachments: SOLR-3141.patch, SOLR-3141.patch
> Background: LUCENE-3454 renames optimize() as forceMerge(). Please read that issue first.
> Now that optimize() is rarely necessary anymore, and renamed in Lucene APIs, what should
be done with Solr's ancient optimize command?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message