lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adriano Crestani <>
Subject Re: The new Contrib QueryParser should not be slated to replace the old one yet
Date Wed, 12 Aug 2009 07:47:30 GMT
Some comments in line:

The new QueryParser has not been used much at all yet. Its interfaces (which
will need to abide by back compat in core) have not been vetted enough.

Agreed, I still think some points must still be discussed about the API, and
to start discussing about it, the contributors must have a deeper understand
about the main points of the new QP, so further we can discuss what must be
adjusted. So, I think it should stay longer on contrib as experimental and
as people start (and they already started [?] ) using it we will have a
feedback from them.

The new parser appears to add complication to some of things that were very
simple with the old parser.

Some things could be easily done with the old QP, because it was designed
specificily for that. For example, you can easily change how RangeQuery
objects are created, because the old QP provides a method for that.
Otherwise, it becomes very difficult to maintain, add extra processing, plug
and unplug functionality, separate syntax from semantic. On the company I
work for, we used to use the old one, just extending it. There was always a
new requirement and the code had to be changed, there was IF statements and
control variables all over the place, I wonder if other companies are
running into the same problems using the old QP. My conclusion here is:
things that are simple are usually not very powerfull, flexible and

This new QP that was contributed to Lucene can be split in 2 parts: core and
lucene QP implementation.

The core contains the QP framework classes, the framework tries to define
the best way to implement and QP, so it's easily mantainable and
extensible/flexible if you follow the rules. Of course the user can just
ignore what the framework suggest, like to do processing at buiding time, or
parsing at processing time and etc, it does not prohibit that. Maybe the
framework classes are not finished yet, they might require more work to get
it in a better shape, at the same time I think it's ready for a release,
since you can already write a complete QP with it. I would like to suggest
that the core classes to be included in 3.0 lucene core.

The lucene QP implementation is the old QP implementation applied against
the new framework rules. The syntax and semantic were separated and now on
any new functionality or change can be easily perfomed. OK, maybe this new
implementation should be in contrib and whether the user thinks it's better
than the old one.

On Tue, Aug 11, 2009 at 10:53 PM, Jason Rutherglen <> wrote:

> With the new QP we can build out a syntax that's compatible with
> GData and be able to embed location/spatial queries directly
> into the query string. (i.e. @+40.75-074.00 + 5mi)
> SQL like range queries (i.e. [megapixel >= 3.0])
> On Tue, Aug 11, 2009 at 10:44 PM, Jason
> Rutherglen<> wrote:
> > I'm starting to use the new parser to emulate Google's queries
> > (i.e. a phrase query with a single term means no-stemming,
> > something the current QP doesn't allow because it converts the
> > quoted query into a term query inside the JavaCC portion). It's
> > been very straightforward and logical to use (so far).
> >
> > Thanks to the contrib query parser team!
> >
> > On Tue, Aug 11, 2009 at 10:54 AM, Mark Miller<>
> wrote:
> >> I don't think we should stick with the current path of replacing the
> current
> >> QueryParser with the new contrib QueryParser in Lucene 3.0.
> >>
> >> The new QueryParser has not been used much at all yet. Its interfaces
> (which
> >> will need to abide by back compat in core) have not been vetted enough.
> >>
> >> The new parser appears to add complication to some of things that were
> very
> >> simple with the old parser.
> >>
> >> The main benefits of the new parser are claimed to be the ability to
> plug
> >> and play many syntaxes and QueryBuilders. This is not an end user
> benefit
> >> though and I'm not even sure how much of a benefit it is to us. There is
> >> currently only one impl. It seems to me, once you start another impl,
> its a
> >> long shot that the exact same query tree representation is going to work
> >> with a completely different syntax. Sure, if you are just doing postfix
> >> rather than prefix, it will be fine – but the stuff that would likely be
> >> done – actual new syntaxes – are not likely to be very pluggable. If a
> >> syntax can map to the same query tree, I think we would likely stick to
> a
> >> single syntax – else suffer the confusion and maintenance headaches for
> >> syntactic sugar. More than a well factored QueryParser that can more
> easily
> >> allow different syntaxes to map to the same query tree representation, I
> >> think we just want a single solid syntax for core Lucene that supports
> Spans
> >> to some degree. We basically have that now, sans the spans support.
> Other,
> >> more exotic QueryParsers should live in contrib, as they do now.
> >>
> >> Which isn't to say this QueryParser should not one day rule the roost –
> but
> >> I don't think its earned the right yet. And I don't think there is a
> hurry
> >> to toss the old parser.
> >>
> >> Personally, I think that the old parser should not be deprecated. Lets
> let
> >> the new parser breath in contrib for a bit. Lets see if anyone actually
> adds
> >> any other syntaxes. Lets see if the pluggability results in any
> >> improvements. Lets see if some of the harder things to do (overriding
> query
> >> build methods?) become easier or keep people from using the new parser.
> >>
> >> Lets just see if the new parser draws users without us forcing them to
> it.
> >> And lets also wait and see what other committers say – not many have
> gotten
> >> much time to deal with the new parser, or deal with user list questions
> on
> >> it.
> >>
> >> I just think its premature to start moving people to this new parser. It
> >> didn't even really get in until right before release – the paint on the
> >> thing still reeks. There is no rush. I saw we undeprecate the current
> >> QueryParser and remove the wording in the new QueryParser about it
> replacing
> >> the new in 3.0. Later, if we think it should replace it (after having
> some
> >> experience to judge from), we can reinstate the current plan. Anyone
> agree?
> >>
> >> --
> >> - Mark
> >>
> >>
> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail:
> >> For additional commands, e-mail:
> >>
> >>
> >
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message