lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <>
Subject Re: The new Contrib QueryParser should not be slated to replace the old one yet
Date Wed, 12 Aug 2009 07:43:28 GMT
> With the new QP we can build out a syntax that's compatible with
> GData and be able to embed location/spatial queries directly
> into the query string. (i.e. @+40.75-074.00 + 5mi)

What do you mean "with the new QP"? What prevents you from doing that w/o
the new QP, as in writing your own QP? What are the benefits the new QP has
when you come to deal w/ such terms? Unless you're talking about extending
the Lucene syntax w/ spatial clauses. Just for my education, how do you
extend the new QP w/ this information? Can you extend the Tokenizer, or do
you need to write a new one?

I'm trying to separate between the query syntax and a QP. The new QP is more
of a framework for how to parse queries. It's well architected and designed.
It allows to build different QPs for different syntaxes easily.

As for the query syntax, what if we had augmented Lucene query syntax w/
opaque clauses support. Something like @qpname::'query string'. Then, we can
add to a QP a QP mapping from qpname to QP instance. That would allow anyone
to use Lucene's QP and write new QPs (however they want) to match different
opaque clauses.

For the example above, I could write this query: "restaurants
+ 5mi' " (quotes are not part of the query string) and instantiate the QP as
QueryParser qp = new QueryParser();
qp.addQueryParser("spatial", new SpatialQueryParser());

Upon parsing, the default QP would hit the opaque clause and defer parsing
of the text in between ' to SpatialQueryParser. We'd need to come up w/ a
simple QP interface, with a parse() method or something that it can call.
Nothing too fancy.

SpatialQueryParser could be implemented however we choose. Not necessarily
using the new QP framework.

Maybe we should add this to Lucene anyway, and the new QP would just make
the implementations easier.

BTW, in case I managed to make a wrong impression - I'm not against the new
QP :).


On Wed, Aug 12, 2009 at 8:53 AM, Jason Rutherglen <> wrote:

> With the new QP we can build out a syntax that's compatible with
> GData and be able to embed location/spatial queries directly
> into the query string. (i.e. @+40.75-074.00 + 5mi)
> SQL like range queries (i.e. [megapixel >= 3.0])
> On Tue, Aug 11, 2009 at 10:44 PM, Jason
> Rutherglen<> wrote:
> > I'm starting to use the new parser to emulate Google's queries
> > (i.e. a phrase query with a single term means no-stemming,
> > something the current QP doesn't allow because it converts the
> > quoted query into a term query inside the JavaCC portion). It's
> > been very straightforward and logical to use (so far).
> >
> > Thanks to the contrib query parser team!
> >
> > On Tue, Aug 11, 2009 at 10:54 AM, Mark Miller<>
> wrote:
> >> I don't think we should stick with the current path of replacing the
> current
> >> QueryParser with the new contrib QueryParser in Lucene 3.0.
> >>
> >> The new QueryParser has not been used much at all yet. Its interfaces
> (which
> >> will need to abide by back compat in core) have not been vetted enough.
> >>
> >> The new parser appears to add complication to some of things that were
> very
> >> simple with the old parser.
> >>
> >> The main benefits of the new parser are claimed to be the ability to
> plug
> >> and play many syntaxes and QueryBuilders. This is not an end user
> benefit
> >> though and I'm not even sure how much of a benefit it is to us. There is
> >> currently only one impl. It seems to me, once you start another impl,
> its a
> >> long shot that the exact same query tree representation is going to work
> >> with a completely different syntax. Sure, if you are just doing postfix
> >> rather than prefix, it will be fine – but the stuff that would likely be
> >> done – actual new syntaxes – are not likely to be very pluggable. If a
> >> syntax can map to the same query tree, I think we would likely stick to
> a
> >> single syntax – else suffer the confusion and maintenance headaches for
> >> syntactic sugar. More than a well factored QueryParser that can more
> easily
> >> allow different syntaxes to map to the same query tree representation, I
> >> think we just want a single solid syntax for core Lucene that supports
> Spans
> >> to some degree. We basically have that now, sans the spans support.
> Other,
> >> more exotic QueryParsers should live in contrib, as they do now.
> >>
> >> Which isn't to say this QueryParser should not one day rule the roost –
> but
> >> I don't think its earned the right yet. And I don't think there is a
> hurry
> >> to toss the old parser.
> >>
> >> Personally, I think that the old parser should not be deprecated. Lets
> let
> >> the new parser breath in contrib for a bit. Lets see if anyone actually
> adds
> >> any other syntaxes. Lets see if the pluggability results in any
> >> improvements. Lets see if some of the harder things to do (overriding
> query
> >> build methods?) become easier or keep people from using the new parser.
> >>
> >> Lets just see if the new parser draws users without us forcing them to
> it.
> >> And lets also wait and see what other committers say – not many have
> gotten
> >> much time to deal with the new parser, or deal with user list questions
> on
> >> it.
> >>
> >> I just think its premature to start moving people to this new parser. It
> >> didn't even really get in until right before release – the paint on the
> >> thing still reeks. There is no rush. I saw we undeprecate the current
> >> QueryParser and remove the wording in the new QueryParser about it
> replacing
> >> the new in 3.0. Later, if we think it should replace it (after having
> some
> >> experience to judge from), we can reinstate the current plan. Anyone
> agree?
> >>
> >> --
> >> - Mark
> >>
> >>
> >>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail:
> >> For additional commands, e-mail:
> >>
> >>
> >
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message