lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <>
Subject Re: Proposal about Version API "relaxation"
Date Wed, 21 Apr 2010 17:56:23 GMT
So basically, API-wise, the stable branch will remain like it is
today: API changes under deprecation path, bw breaks as long as they
are documented in CHANGES etc. Trunk will be allowed to change the API
as it sees fit (but still document the changes in CHANGES).

Index-format wise, we adopt Doron's proposal of the 3 support levels
for trunk (for stable it's always L1).

The only downside is that we will need to do everything twice: once on
stable and once on trunk. I still think that most of the issues and
development don't affect bw at all and thus we'll always say "this
needs to go to stable and trunk" which will just be an annoyance and
complicate the life of the developers even more because not only will
we need to keep bw compat, we'll need to write the code for trunk as

What if we always develop on trunk, release it more often, and if
requested or a committer needs it, we backport a certain feature to
stable? That way, stable includes really what's been specifically
needed and trunk gets the latest and greatest API and features set.
Since we still keep index format bw (pending levels) most apps should
not have any problem upgrading to a latest major, given that they
adapt to the new API …


On Wednesday, April 21, 2010, Michael McCandless
<> wrote:
> Trying to summarize what we seem to be roughly converging to, here:
>   * Up front: consolidate all Solr core, Lucene core, contrib
>     analyzers into one place (contrib/analyzers).  Don't use Version
>     in there; instead, the released JAR is versioned.  The app picks
>     its required version compatibility by picking the right analyzers
>     JAR to use.
>   * Switch to two active branches for ongoing development (stable &
>     trunk) -- stable gets features/bug fixes that are low risk / don't
>     change (too many?) APIs.  We make minor releases off of stable
>     (3.0, 3.1, 3.2, and possibly also bug-fix only .Z release like
>     3.0.1), while trunk has ongoing non-back-compatible changes.
>     Development should be active on both.
>   * Maybe release 3.1 today, by branching off trunk before flex
>     landed, maybe minus a few changes.  This would be the start of the
>     stable branch for 3.x releases, and trunk becomes experimental.
>   * Index compatibility on a major release falls into 3 levels:
>      - Level 1: index is read/written "live" (this is what we have
>        today).
>      - Level 2: we provide a migration tool (this is what we'd do for
>        the flex changes), to carry the index forward.
>      - Level 3: app must re-index.
>     I would expect level 3 to be very rare.... (it's never happened so
>     far!).
> Does this sound right?  Objections?
> Mike
> On Mon, Apr 19, 2010 at 4:37 AM, Shai Erera <> wrote:
>> The 'unless' part is good and in place IMO. Certainly, if sometimes in the
>> future Lucene moves away from segmented indexing approach into something
>> else, I wouldn't expect a migration tool to be introduced. So overhauling
>> index file format might be ok to go w/o any migration tool introduced.
>> But I think we tend to take it as a "all or nothing" deal. When the index
>> file format changed (and Mike can correct me if I'm wrong), it usually
>> didn't introduce such overhauling changes. For example, "flex scoring" - we
>> can say that a flex-scoring index should read older indexes, and if one did
>> not want to take advantage of that feature, one should not reindex just
>> because he upgrade to Lucene 4.0, for other reasons. I believe that should
>> be relatively easy to support.
>> And I think that people understand that they cannot take advantage of a new
>> feature until they reindex (features like flex-scoring, numeric queries
>> etc.) -- I just think we shouldn't *force* them to reindex just because the
>> indexing code now 'expects' those files/terms to be in place for regular
>> indexing behavior (unrelated to those advanced features).
>> I'm +1 for that proposal.
>> Shai
>> On Mon, Apr 19, 2010 at 11:28 AM, Doron Cohen <> wrote:
>>> Late joining... could we agree on an "intention" to provide an index
>>> migration tool when/if format back comp. has to be broken? It is not clear
>>> to me that this was agreed... So here is a suggestion for a revised index
>>> format backwards compatibility policy:
>>> Starting release 4.0, Lucene has a limited file formats back-compatibility
>>> between major versions, falling into one of the three possible levels:
>>> (Level 1) When possible, Version X is would be able to read indexes
>>> generated by any X-1 version after and including version X-1.0. (Level 2) If
>>> version X cannot read indexes of version X-1, the release of version X would
>>> be accompanied by a tool for migrating indexes from X-1 to X, unless (Level
>>> 3) the nature of the specific change does not allow for the development of
>>> such a migration tool. For the exact level of file back compatibility of a
>>> release see the specific release notes.
>>> Not sure if the "unless" part (no. 3) would ever materialize, but I think
>>> it provides a required freedom.
>>> Doron
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message