accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Boyd <db...@incadencecorp.com>
Subject Problem with IntersectingIterator and IndexDocIterator not returning results.
Date Thu, 26 May 2016 14:47:40 GMT
All:

    I am using accumulo 1.6.1.  I am using a sharded index to search for 
data
that matches the values of certain fields.  Here is the situation:

I have four fields EntityId, EntityIdType, EntityName, EntitySource.

Sometimes I need all records which match EntityId and EntityIdType
Othertimes I need all records which match all four fields.

The plan was use uses ranges in the scanner to determine which fields to
match against.  I have tried both subsets of ranges, setting all ranges, and
gotten the same result.

I created an Index as follows:
RowId = fieldname
ColumnFamily = fieldvalue
ColumnQualifier = the overall record id (RowID) of my main record in 
another table.

Here is the output of a scan of my index table:

> entityid 
> 1707945d-34d8-455d-85b1-55610739ce62:1707945d-34d8-455d-85b1-55610739ce62 
> []
> entityidtype GUID:1707945d-34d8-455d-85b1-55610739ce62 []
> name TestEntity:1707945d-34d8-455d-85b1-55610739ce62 []
> source Unit Test:1707945d-34d8-455d-85b1-55610739ce62 []

NOTE:  While in this case the entityid equals the overall RowID from the 
other table that is not always true

When I run the code below it does not return any rows in the scanner.
In the debugger when running the code below terms show as follows:
[1707945d-34d8-455d-85b1-55610739ce62, GUID, TestEntity, Unit Test]

I have tried both IntersectingIterator and IndexDocIterator both have 
the same results.
For whatever reason the API docs for these classes is not showing up on 
the Apache
Accumulo site.

Am I missing the purpose/function of this iterator?

Do I have to call IndexedDocIterator.setColfs with some values so I get 
the column qualifiers back?

Below is my code:

public List<String> getCoalesceEntityKeysForEntityId(String entityId,
                                                          String 
entityIdType,
                                                          String entityName,
                                                          String 
entitySource) throws CoalescePersistorException
     {
         // Use are sharded term index to find the merged keys
         Connector dbConnector = null;

         ArrayList<String> keys = new ArrayList<String>();

         Text[] terms = {new Text(entityId), new Text(entityIdType),
                 new Text(entityName), new Text(entitySource)};


         try {
             dbConnector = AccumuloDataConnector.getDBConnector();

             BatchScanner keyscanner = 
dbConnector.createBatchScanner(AccumuloDataConnector.coalesceEntityIndex, 
Authorizations.EMPTY, 4);

             // Set up an IntersectingIterator for the values
             IteratorSetting iter = new IteratorSetting(1, "intersect", 
IndexedDocIterator.class);
             IndexedDocIterator.setColumnFamilies(iter,terms);
             keyscanner.addScanIterator(iter);

             // Use ranges to limit the bins searched
             //ArrayList<Range> ranges = new ArrayList<Range>();
             // May not be necessary to restrict ranges but will do it 
to be safe
             //ranges.add(new Range("entityid"));
             //ranges.add(new Range("entityitype"));
             //ranges.add(new Range("entityname"));
            // ranges.add(new Range("source"));
             //keyscanner.setRanges(ranges);
             keyscanner.setRanges(Collections.singleton(new Range()));

             // Return the list of keys
             for(Entry<Key,Value> entry : keyscanner) {
keys.add(entry.getKey().getColumnQualifier().toString());
             }

         } catch (TableNotFoundException ex) {
             System.err.println(ex.getLocalizedMessage());
             return null;
         }

         return keys;
     }



-- 
========= mailto:dboyd@incadencecorp.com ============
David W. Boyd
VP,  Data Solutions
10432 Balls Ford, Suite 240
Manassas, VA 20109
office:   +1-703-552-2862
cell:     +1-703-402-7908
============== http://www.incadencecorp.com/ ============
ISO/IEC JTC1 WG9, editor ISO/IEC 20547 Big Data Reference Architecture
Chair ANSI/INCITS TC Big Data
Co-chair NIST Big Data Public Working Group Reference Architecture
First Robotic Mentor - FRC, FTC - www.iliterobotics.org
Board Member- USSTEM Foundation - www.usstem.org

The information contained in this message may be privileged
and/or confidential and protected from disclosure.
If the reader of this message is not the intended recipient
or an employee or agent responsible for delivering this message
to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited.  If you have received this communication
in error, please notify the sender immediately by replying to
this message and deleting the material from any computer.

  


Mime
View raw message