Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of karl.wright@nokia.com
 designates 192.100.122.230 as permitted sender)
From: <karl.wright@nokia.com>
To: <dev@lucene.apache.org>, <connectors-dev@incubator.apache.org>
Date: Thu, 29 Apr 2010 01:32:06 +0200
Subject: RE: Solr query question
Thread-Topic: Solr query question
Thread-Index: AcrnHYKGLW9efMzZSHmH0W4C/sSTkQABTlWmAAHhaKo=
Message-ID: 
 <CF3CE3EFBCA3564185DF065952A267C8530507F259@NOK-EUMSG-01.mgdnok.nokia.com>
References: 
 <CF3CE3EFBCA3564185DF065952A267C8530507F253@NOK-EUMSG-01.mgdnok.nokia.com>
 <CF3CE3EFBCA3564185DF065952A267C8530507F256@NOK-EUMSG-01.mgdnok.nokia.com>,<37BA4DF7-BF1A-4538-8F46-5F08714028DD@gmail.com>,<CF3CE3EFBCA3564185DF065952A267C8530507F257@NOK-EUMSG-01.mgdnok.nokia.com>
In-Reply-To: 
 <CF3CE3EFBCA3564185DF065952A267C8530507F257@NOK-EUMSG-01.mgdnok.nokia.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

I tried the getFilters() approach.  It turned out I also needed to create a=
 list and do setFilters() if getFilters() returns null, but that was easily=
 remedied.  When this is done, it once again works fine if the component is=
 added "last".  But if it is added "first", we now get a stack trace from s=
omeplace not in my search component:

null

java.lang.NullPointerException
	at org.apache.lucene.search.FilteredQuery.hashCode(FilteredQuery.java:229)
	at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
	at org.apache.solr.common.util.ConcurrentLRUCache.get(ConcurrentLRUCache.j=
ava:87)
	at org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:133)
	at org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.ja=
va:538)
	at org.apache.solr.search.SolrIndexSearcher.getDocSet(SolrIndexSearcher.ja=
va:581)
	at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher=
.java:903)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.=
java:884)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:=
341)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent=
.java:178)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(Searc=
hHandler.java:195)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandler=
Base.java:131)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1321)
	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.j=
ava:341)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.=
java:244)
	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHa=
ndler.java:1089)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365=
)
	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:=
216)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181=
)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712=
)
	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandle=
rCollection.java:211)
	at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.ja=
va:114)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139=
)
	at org.mortbay.jetty.Server.handle(Server.java:285)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConn=
ection.java:821)
	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
	at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.ja=
va:226)
	at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.j=
ava:442)

So, I think there's got to be some assumptions being made in one or more of=
 the default search components that's making it impossible to add a filter =
before it gets control.  Or maybe I'm just tripping over a bug somewhere.

(FWIW, the version of Solr I am using is a nightly from mid-March.)

Karl

________________________________________
From: Wright Karl (Nokia-S/Cambridge)
Sent: Wednesday, April 28, 2010 6:40 PM
To: dev@lucene.apache.org; connectors-dev@incubator.apache.org
Subject: RE: Solr query question

Adding to the getFilters() list seems reasonable - although, to be fair, my=
 code does seem to work as intended when the component is added "last".  I'=
ll do some experimentation and see what model things work most consistently=
 with.

TermRangeQuery doesn't seem to map readily to the functionality I'm looking=
 for.  I basically want to match documents that have no values for a specif=
ic field.  TermRange implies that I know the potential set of values, and I=
 don't at that point (unless I can use null or something for the start/end =
strings in the range?)  Plus it is a Query, not a Filter, so if I use it I'=
d also need to wrap it with FilterWrappedQuery or some such, no?  Is there =
any benefit in using Query objects over Filter objects, or visa versa?  (I =
*am* trying to be sure that the security filter does not affect relevance, =
for what it's worth...)

Karl

________________________________________
From: ext Erik Hatcher [erik.hatcher@gmail.com]
Sent: Wednesday, April 28, 2010 5:54 PM
To: connectors-dev@incubator.apache.org
Cc: dev@lucene.apache.org
Subject: Re: Solr query question

Rather than rewriting the original query, add a filter query (fq param
on the HTTP interface).  I think in the API you'll be using
rb.getFilters() and adding a filter to List returned.

Running your component last won't work (will it?), as it needs to be
run before the "query" component to take effect.

Re: WildcardFilter - I think you want TermRangeQuery there instead.

        Erik


On Apr 28, 2010, at 5:35 PM, <karl.wright@nokia.com> wrote:

> Turns out that, for the standard requestHandler, running this
> SearchComponent first causes its rewritten query to be lost.
> Running last fixed the problem.  (I'd *love* to know why that would
> be necessary.)
>
> But I'd still like comment as to whether the WildcardFilter
> construct is expected to be efficient in this context, or not. ;-)
>
> Karl
>
>
> ________________________________________
> From: Wright Karl (Nokia-S/Cambridge)
> Sent: Wednesday, April 28, 2010 2:57 PM
> To: connectors-dev@incubator.apache.org
> Subject: Solr query question
>
> Hi Solr-knowledgeable folks,
>
> The LCF Solr SearchComponent plugin I'm developing doesn't quite
> work.  The query I'm trying to do is:
>
> -(allow_token_document:*) and -(deny_token_document:*) and <the
> user's search>
>
> The result I'm seeing is that everything in the user's search
> matches, unlike what I see in the admin UI, where the above query
> works perfectly.
>
> The code I'm using to do the negative wildcard searches is as follows:
>
>  public void prepare(ResponseBuilder rb) throws IOException
>  {
>      BooleanFilter bf =3D new BooleanFilter();
>
>
>      // No authenticated user name; only return 'public' documents
> (those with no security tokens at all)
>      // That query is:
>      // (fieldAllowShare is empty AND fieldDenyShare is empty AND
> fieldAllowDocument is empty AND fieldDenyDocument is empty)
>
>      // We're trying to map to:  -(fieldAllowShare:*) , which should
> be pretty efficient in Solr because it is negated.  If this turns
> out not to be so, then we should
>      // have the SolrConnector inject a special token into these
> fields when they otherwise would be empty, and we can trivially
> match on that token.
>
>      bf.add(new FilterClause(new WildcardFilter(new
> Term(fieldAllowDocument,"*")),BooleanClause.Occur.MUST_NOT));
>      bf.add(new FilterClause(new WildcardFilter(new
> Term(fieldDenyDocument,"*")),BooleanClause.Occur.MUST_NOT));
>
>     // Concatenate with the user's original query.
>     FilteredQuery query =3D new FilteredQuery(rb.getQuery(),bf);
>     rb.setQuery(query);
>  }
>
>
> Any hints welcome!
> Karl


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org