lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-1913) QParserPlugin plugin for Search Results Filtering Based on Bitwise Operations on Integer Fields
Date Thu, 27 Sep 2012 14:38:07 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464779#comment-13464779
] 

Lars edited comment on SOLR-1913 at 9/28/12 1:37 AM:
-----------------------------------------------------

I implemented the BitwiseQueryParserPlugin as described above. Our solr 3.6.1 is still working.
Now I need to implement the bitwise operator into the query, but I don't know how.

What I want to do is:

A document has a bitmask which defines the access permissions a user has. The search should
only return the results where the document bit are set in the users bitmask. In pseudo code
it looks like:
{code}document.bitmask == (document.bitmask BITWISE_AND users.bitmask){code}
Could this be implemented with the BitwiseQueryParserPlugin? It's even a bit mor complicated,
but maybe I'll figure out the rest on my own.

What I tried to now:

Get all documents which maches exactly the bitmask ... this works well, without bitwise operations:

http://solr:8080/solr/select?qf=t_title&fl=*,score&fq=is_bitmask:1234

Next step: Use the bitwise operator:

http://solr:8080/solr/select?qf=t_title&fl=*,score&fq={!bitwise field=is_bitmask op=AND
source=1234}*

This fails. The Message from error log:

{code}
Sep 27, 2012 8:57:41 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select params={qf=t_title&fl=*,score&fq={!bitwise+field%3Dis_bitmask+op%3DAND+source%3D1234}}
status=500 QTime=15 
Sep 27, 2012 8:57:41 AM org.apache.solr.common.SolrException log
SEVERE: java.lang.NullPointerException
  at org.apache.lucene.search.FilteredQuery.hashCode(FilteredQuery.java:268)
  at java.util.AbstractList.hashCode(AbstractList.java:542)
  at org.apache.solr.search.QueryResultKey.<init>(QueryResultKey.java:49)
  at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1084)
  at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:375)
  at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:394)
  at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:186)
  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
  at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
  at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
  at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
  at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
  at java.lang.Thread.run(Thread.java:636)
{code}

schema.xml defines the bitmask field as 

<dynamicField name="is_*" type="tlong" multiValued="false" termVectors="true" />

... while tlong means:

<fieldType name="tlong" class="solr.TrieLongField" indexed="true" stored="true" precisionStep="8"
omitNorms="true" positionIncrementGap="0"/>

... instead of tlog I also tried with integer:

<fieldType name="integer" class="solr.IntField" omitNorms="true"/>

but same result.

I use the [schema.xml|http://drupalcode.org/project/search_api_solr.git/blob/3e6fda7:/solrconfig.xml]
provided by the drupal module [search_api_solr|http://drupal.org/project/search_api_solr]

Any suggestions? I'm not a java developer and also new to solr.

@Arnaud: Did you figure out how to chain/combine the filters?
                
      was (Author: osopolar):
    I implemented the BitwiseQueryParserPlugin as described above. Our solr 3.6.1 is still
working. Now I need to implement the bitwise operator into the query, but I don't know how.

What I want to do is:

A document has a bitmask which defines the access permissions a user has. The search should
only return the results where the document bit are set in the users bitmask. In pseudo code
it looks like:
{code}document.bitmask == (document.bitmask BITWISE_AND users.bitmask){code}
Could this be implemented with the BitwiseQueryParserPlugin? It's even a bit mor complicated,
but maybe I'll figure out the rest on my own.

What I tried to now:

Get all documents which maches exactly the bitmask ... this works well, without bitwise operations:

http://solr:8080/solr/select?qf=t_title&fl=*,score&fq=is_bitmask:1234

Next step: Use the bitwise operator:

http://solr:8080/solr/select?qf=t_title&fl=*,score&fq={!bitwise field=is_bitmask op=AND
source=1234}*

This fails. The Message from error log:

{code}
Sep 27, 2012 8:57:41 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/select params={qf=t_title&fl=*,score&fq={!bitwise+field%3Dis_bitmask+op%3DAND+source%3D1234}}
status=500 QTime=15 
Sep 27, 2012 8:57:41 AM org.apache.solr.common.SolrException log
SEVERE: java.lang.NullPointerException
  at org.apache.lucene.search.FilteredQuery.hashCode(FilteredQuery.java:268)
  at java.util.AbstractList.hashCode(AbstractList.java:542)
  at org.apache.solr.search.QueryResultKey.<init>(QueryResultKey.java:49)
  at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1084)
  at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:375)
  at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:394)
  at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:186)
  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
  at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
  at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
  at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
  at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
  at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
  at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
  at java.lang.Thread.run(Thread.java:636)
{code}

schema.xml defines the bitmask field as 

<dynamicField name="is_*" type="tlong" multiValued="false" termVectors="true" />

... while tlong means:

<fieldType name="tlong" class="solr.TrieLongField" indexed="true" stored="true" precisionStep="8"
omitNorms="true" positionIncrementGap="0"/>

... instead of tlog I also tried with integer:

<fieldType name="integer" class="solr.IntField" omitNorms="true"/>

but same result.

I use the schema.xml provided by the drupal module [search_api_solr|http://drupal.org/project/search_api_solr]

Any suggestions? I'm not a java developer and also new to solr.

@Arnaud: Did you figure out how to chain/combine the filters?
                  
> QParserPlugin plugin for Search Results Filtering Based on Bitwise Operations on Integer
Fields
> -----------------------------------------------------------------------------------------------
>
>                 Key: SOLR-1913
>                 URL: https://issues.apache.org/jira/browse/SOLR-1913
>             Project: Solr
>          Issue Type: New Feature
>          Components: search
>            Reporter: Israel Ekpo
>             Fix For: 4.1
>
>         Attachments: bitwise_filter_plugin.jar, SOLR-1913.bitwise.tar.gz
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> BitwiseQueryParserPlugin is a org.apache.solr.search.QParserPlugin that allows 
> users to filter the documents returned from a query
> by performing bitwise operations between a particular integer field in the index
> and the specified value.
> This Solr plugin is based on the BitwiseFilter in LUCENE-2460
> See https://issues.apache.org/jira/browse/LUCENE-2460 for more details
> This is the syntax for searching in Solr:
> http://localhost:8983/path/to/solr/select/?q={!bitwise field=fieldname op=OPERATION_NAME
source=sourcevalue negate=boolean}remainder of query
> Example :
> http://localhost:8983/solr/bitwise/select/?q={!bitwise field=user_permissions op=AND
source=3 negate=true}state:FL
> The negate parameter is optional
> The field parameter is the name of the integer field
> The op parameter is the name of the operation; one of {AND, OR, XOR}
> The source parameter is the specified integer value
> The negate parameter is a boolean indicating whether or not to negate the results of
the bitwise operation
> To test out this plugin, simply copy the jar file containing the plugin classes into
your $SOLR_HOME/lib directory and then
> add the following to your solrconfig.xml file after the dismax request handler:
> <queryParser name="bitwise" class="org.apache.solr.bitwise.BitwiseQueryParserPlugin"
basedOn="dismax" />
> Restart your servlet container.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message