lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: New Token API was Re: Payloads and TrieRangeQuery
Date Mon, 15 Jun 2009 16:19:29 GMT
I thought the primary goal of switching to AttributeSource (yes, the
name is very generic...) was to allow extensibility to what's created
per-Token, so that an app could add their own attrs without costly
subclassing/casting per Token, independent of other other "things"
adding their tokens, etc.

EG, trie* takes advantage of this extensibility by adding a
ShiftAttribute.

Subclassing Token in your app wasn't a good solution for various
reasons.

I do think the API is somewhat more cumbersome than before, and I
don't like that about it (consumability!).

But net/net I think the change is good, and it's one of the
baby steps for flexible indexing (bullet #11):

  http://wiki.apache.org/jakarta-lucene/Lucene2Whiteboard

Ie it addresses the flexibility during analysis.

I don't think anything was "held back" in this effort. Grant, are you
referring to LUCENE-1458?  That's "held back" simply because the only
person working on it (me) got distracted by other things to work on.

Flexible indexing (all of bullet #11) is a complex project, and we
need to break it into baby steps like this one.  We've already made
good progress on it: you can already make custom attrs and a custom
(but, package private) indexing chain if you want.

Next step is pluggable codecs for writing index files (LUCENE-1458),
and APIs for reading them (that generalize Terms/TermDoc/TermPositions
we have today).

Mike

On Sun, Jun 14, 2009 at 11:41 PM, Shai Erera<serera@gmail.com> wrote:
> The "old" API is deprecated, and therefore when we release 2.9 there might
> be some people who'd think they should move away from it, to better prepare
> for 3.0 (while in fact this many not be the case). Also, we should make sure
> that when we remove all the deprecations, this will still exist (and
> therefore, why deprecate it now?), if we think this should indeed be kept
> around for at least a while longer.
>
> I personally am all for keeping it around (it will save me a huge
> refactoring of an Analyzer package I wrote), but I have to admit it's only
> because I've got quite comfortable with the existing API, and did not have
> the time to try the new one yet.
>
> Shai
>
> On Mon, Jun 15, 2009 at 3:49 AM, Mark Miller <markrmiller@gmail.com> wrote:
>>
>> Mark Miller wrote:
>>>
>>> I don't know how I feel about rolling the new token api back.
>>>
>>> I will say that I originally had no issue with it because I am very
>>> excited about Lucene-1458.
>>>
>>> At the same time though, I'm thinking Lucene-1458 is a very advanced
>>> issue that will likely be for really expert usage (though I can see benefits
>>> falling to general users).
>>>
>>> I'm slightly iffy about making an intuitive api much less intuitive for
>>> an expert future feature that hasn't fully materialized in Lucene yet. It
>>> almost seems like that fight should weigh towards general usage and standard
>>> users.
>>>
>>> I don't have a better proposal though, nor the time to consider it at the
>>> moment. I was just more curious if anyone else had any thoughts. I hadn't
>>> realized Grant had asked a similar question not long ago
>>> with no response. Not sure how to take that, but I'd think that would
>>> indicate less problems with people than more. On the other hand, you don't
>>> have to switch yet (with trunk) and we have yet to release it. I wonder how
>>> many non dev, every day users have really had to tussle with the new API
>>> yet. Not many people complaining too loudly at the moment though.
>>>
>>> Asking for a roll back seems a bit extreme without a little more support
>>> behind it than we have seen.
>>>
>>> - Mark
>>
>> PS
>>
>> I know you didnt ask for a rollback Grant - just kind of talking in a
>> general manner. I see your point on getting the search side in, I'm just not
>> sure I agree that it really matters if one hits before the other. Like Mike
>> says, you don't
>> have to switch to the new API yet.
>>
>> --
>> - Mark
>>
>> http://www.lucidimagination.com
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message