lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Israel Ekpo <israele...@gmail.com>
Subject Bitwise Operations on Integer Fields in Lucene and Solr Index
Date Thu, 13 May 2010 21:27:08 GMT
Hello Lucene and Solr Community

I have a custom org.apache.lucene.search.Filter that I would like to
contribute to the Lucene and Solr projects.

So I would need some direction as to how to create and ISSUE or submit a
patch.

It looks like there have been changes to the way this is done since the
latest merge of the two projects (Lucene and Solr).

Recently, some Solr users have been looking for a way to perform bitwise
operations between and integer value and some fields in the Index

So, I wrote a Solr QParser plugin to do this using a custom Lucene Filter.

This package makes it possible to filter results returned from a query based
on the results of a bitwise operation on an integer field in the documents
returned from the pre-constructed query.

You can perform three basic types of operations on these integer fields

    * BitwiseOperation.BITWISE_AND (bitwise AND)
    * BitwiseOperation.BITWISE_OR (bitwise inclusive OR)
    * BitwiseOperation.BITWISE_XOR (bitwise exclusive OR)

You can also negate the results of these operations.

For example, imagine there is an integer field in the index named "flags"
with the a value 8 (1000 in binary). The following results will be expected
:

   1. A source value of 8 will match during a BitwiseOperation.BITWISE_AND
operation, with negate set to false.
   2. A source value of 4 will match during a BitwiseOperation.BITWISE_AND
operation, with negate set to true.

The BitwiseFilter constructor accepts the following values

    * The name of the integer field (A string)
    * The BitwiseOperation object. Example BitwiseOperation.BITWISE_XOR
    * The source value (an integer)
    * A boolean value indicating whether or not to negate the results of the
operation
    * A pre-constructed org.apache.lucene.search.Query

Here is an example of how you would use it with Solr

http://localhost:8983/solr/bitwise/select/?q={!bitwisefield=user_permissions
op=AND source=3 negate=true}state:FL

http://localhost:8983/solr/bitwise/select/?q={!bitwisefield=user_permissions
op=AND source=3}state:FL

Here is an example of how you would use it with Lucene

public class BitwiseTestSearch extends BitwiseTestBase {

    public BitwiseTestSearch()
    {

    }

    public void search() throws IOException, ParseException
    {
        setupSearch();

        // term
        Term t = new Term(COUNTRY_KEY, "us");

        // term query
        Query q = new TermQuery(t);

        // maximum number of documents to display
        int limit = 1000;

        int sourceValue = 0 ;

        boolean negate = false;

        BitwiseFilter bitwiseFilter = new BitwiseFilter(USER_PERMS_KEY,
BitwiseOperation.BITWISE_XOR, sourceValue, negate, q);

        Query fq = new FilteredQuery(q, bitwiseFilter);

        ScoreDoc[] hits = isearcher.search(fq, null, limit).scoreDocs;

        BitwiseResultFilter resultFilter = bitwiseFilter.getResultFilter();

        for (int i = 0; i < hits.length; i++) {

            Document hitDoc = isearcher.doc(hits[i].doc);

            System.out.println(FIRST_NAME_KEY + " field has a value of " +
hitDoc.get(FIRST_NAME_KEY));
            System.out.println(LAST_NAME_KEY + " field has a value of " +
hitDoc.get(LAST_NAME_KEY));
            System.out.println(ACTIVE_KEY + " field has a value of " +
hitDoc.get(ACTIVE_KEY));

            System.out.println(USER_PERMS_KEY + " field has a value of " +
hitDoc.get(USER_PERMS_KEY));

            System.out.println("doc ID --> " + hits[i].doc);


System.out.println("...............................................................");
        }

        System.out.println("sourceValue = " + sourceValue + ",operation = "
+ resultFilter.getOperation().getOperationName() + ", negate = " + negate);

        System.out.println("A total of " + hits.length + " documents were
found from the search\n");

        shutdown();
    }

    public static void main(String args[]) throws IOException,
ParseException
    {
        BitwiseTestSearch search = new BitwiseTestSearch();

        search.search();
    }
}

Any guidance would be highly appreciated.

Thanks.


-- 
"Good Enough" is not good enough.
To give anything less than your best is to sacrifice the gift.
Quality First. Measure Twice. Cut Once.
http://www.israelekpo.com/

Mime
View raw message