Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of dmsmith555@gmail.com designates
 74.125.92.25 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:from:to:in-reply-to:content-type
         :content-transfer-encoding:mime-version:subject:date:references
         :x-mailer;
        b=YS/bW4ioaEUC0Gtk22Xoo/dgay47wHplrn4wItuFh+u/6LSTGljElMuNj82JseHvdW
         HkrFgBrsOOZJApeYfUaTIFA8BE8w1G8qhaw8A8yuE8UDGFAO9WUUOnr+MlDRPMxxecO9
         fXro+5buvj0pKjJ8YP3Sk4eXANgf0zHhBpxQc=
Message-Id: <D1541351-05D9-41CB-973F-27958E1425BC@gmail.com>
From: DM Smith <dmsmith555@gmail.com>
To: java-dev@lucene.apache.org
In-Reply-To: <9ac0c6aa0905190445y5192b144n604336f0323f088f@mail.gmail.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: Lucene's default settings & back compatibility
Date: Tue, 19 May 2009 08:26:13 -0400
References: <9ac0c6aa0905181406l5c951016k97a16d8db766716e@mail.gmail.com>
 <8f0ad1f30905182031l69b2f54ex784fcf8043244f10@mail.gmail.com>
 <F7D7E766-CE8B-4A09-86DD-686FD399EA0A@gmail.com>
 <9ac0c6aa0905190445y5192b144n604336f0323f088f@mail.gmail.com>


On May 19, 2009, at 7:45 AM, Michael McCandless wrote:

> On Tue, May 19, 2009 at 6:47 AM, DM Smith <dmsmith555@gmail.com>  
> wrote:
>
>> It is common in my application, a Bible program, that indexes each  
>> verse
>> (think of a verse as a numbered sentence) as a separate document.  
>> We index
>> everything, including words that are typically stop words as those  
>> might be
>> important to our end users. Besides this, the top 280 word roots  
>> represent
>> 90% of the occurrences.
>> And on searches, we return everything in book order, unless the  
>> user wants
>> to score the result. In that case, we return a small, user  
>> configurable
>> amount of hits ordered by score.
>
> The ability to turn off scoring when sorting by field, new in 2.9,
> should be a good performance boost for your use case (if performance
> is important).
>
>> And we are using Lucene out of the box for the most part. We've  
>> deviated
>> only to incrementally solve performance problems.
>
> Right, my impression is most people will stick w/ Lucene's defaults,
> incrementally changing only limited settings they come across, which
> is why selecting good defaults is vital to Lucene's growth/adoption
> (new users especially simply start w/ our defaults).
>
> But we can't pick good defaults when we're so heavily bound by back- 
> compat.
>
> Which is why I find the Settings approach so appealing :)  Suddenly,
> on all improvements to Lucene, we have the freedom to change our
> defaults so a new user sees all such improvements.

 From my perspective as a user:
Backward compatibility is important, but it is not a be-all and end-all.

To me, if I can drop in the new jar and get bug fixes that's great. My  
expectation is that searches against an existing index will still  
return the same or, in the case of bug fixes, better results.

What I need to know is when that is not the case. Today, we use a  
naming convention of the Lucene jars to indicate whether that is true.  
I'd be just as happy if there were a compatibility level that I could  
check (I'm having to do that in our code as I change our analyzers  
frequently enough to be embarrassed).

The problem, which might be addressed in the "fixing" of core vs  
contrib, is that we use lots of contrib (analyzers, snowball,  
highlighting) and want it to maintain backward compatibility too. (I'm  
happy that has been the case!) So, perhaps a compatibility level per  
contribution.

The packagers for jpackage consider nearly every release of Lucene to  
break backward compatibility, because they treat Lucene as a whole.  
Perhaps that is the same with other Linux distributions. But because  
backward compatibility does not apply to contrib in a strict fashion,  
one cannot reliably use Lucene from distributions unless such a policy  
is the case.

In any case, I don't think anyone should just drop in a new jar  
without some testing. At a minimum, they should compile with  
deprecations turned on.

Regarding deprecations, I'd also be just as happy if a method was marked
	@deprecated This behavior <b>has</b> changed in with this release,  
2.4.3.
That is, as a warning of changed behavior.

And then on the 3.0 release the warning could be removed.

But then again, my use of Lucene, while very important to my  
application, is very simple and easy to change.

-- DM


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org