Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Date: Tue, 30 Nov 2010 17:10:45 -0800
To: dev@lucene.apache.org
Subject: Re: deprecating Versions
Message-ID: <20101201011045.GA12886@rectangular.com>
References: <AANLkTime=giKjdHF+QnP+YpNCLqt4MQWDqzvhjufUP1c@mail.gmail.com>
 <AANLkTi=PH2-GPgUGpJ9yA4J6zLP2FoqEGtP9SXk504xb@mail.gmail.com>
 <AANLkTimHKOsHAZGK20buDe=F_BS1BdSfaU1=XcmXxvuX@mail.gmail.com>
 <AANLkTi=VhKxD3jNh1DgqtxfEYZQ1kF4AUGroQ6qvq-u4@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <AANLkTi=VhKxD3jNh1DgqtxfEYZQ1kF4AUGroQ6qvq-u4@mail.gmail.com>
User-Agent: Mutt/1.5.13 (2006-08-11)
From: Marvin Humphrey <marvin@rectangular.com>

On Mon, Nov 29, 2010 at 05:34:27AM -0500, Robert Muir wrote:
> Is it somehow possible i could convince everyone that all the analyzers we
> provide are simply examples?  This way we could really make this a bit more
> reasonable and clean up a lot of stuff.

I understand what you're getting at.  We don't really expect people to fork an
analyzer code base, though -- so we need to draw a line between e.g. the code
that implements StopFilter and stoplist content.   We want the low-level code
to be part of the library, but maybe we want stoplist content to be considered
example code.

> Seems like we really want to move towards a more declarative model where
> these are just config files... so only then it will ok for us to change them
> because they suddenly aren't suffixed with .java?!

Consider how this might work with e.g. RussianAnalyzer.  The
declaratively-expressed sample analyzer config could contain a hard-coded list
of Russian stop words, and as this hard-coded stoplist would travel with the
index in a config file, there would be no index compatibility problems upon
upgrading Lucene.  The stoplist in the sample config could change, even on
bugfix releases.

Config file syntax would potentially be affected by a Lucene upgrade, but that
doesn't affect index content and maintaining back compat is straightforward.

Things are more difficult with versioning e.g. stemmers, but I think the
stoplist example illustrates the potential of declarative analyzer
specification.  Maybe specifying Version in a sample file and dispatching to
different revs of a Snowball stemmer is less painful than forcing a user to
figure out Version from API documentation?

Having to extract an Analyzer from an index directory does present the
potential for Analyzer mismatches in a multi-node setup where e.g. the machine
that parses the query string and the machine which executes matching are not
the same.

Marvin Humphrey


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org