accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Blum" <>
Subject RE: Querying Accumulo Data Using Java
Date Thu, 28 Apr 2016 16:59:50 GMT
You can’t pass objects because the iterator stack will be run in a different JVM; anything
that isn’t a string or coercible to a string would need to be serialized. So if you have
something complex you will need to serialize it into a string somehow and then deserialize
it in your filter class, which is quite doable.


From: Ben Craig [] 
Sent: Thursday, April 28, 2016 12:54 PM
Subject: Re: Querying Accumulo Data Using Java


Hey Josh,

Thanks for the response.  I was getting pretty lost trying to use the Internal Iterator. 
I will try using the fetchColumn on the scanner.  I guess I have one last question is there
any possible way to pass a java object to a custom filter or are we limited to the PropertyMap
of <String,String> ?  I think its the String,String will work in almost all cases I
need to do but am more curious than anything.





On Thu, Apr 28, 2016 at 1:34 PM, Josh Elser <> wrote:

Hi Ben,

Looks like you're on the right track. Iterator priorities are a little obtuse at first glance;
you probably want to change the 1 to 15 (we can touch on the "why" later).

As far as Iterators/Filters that you should feel comfortable using, check out:

The "system" package are not meant for public consumption (and often are used to implement
internal functionality). This is probably why you're having a hard time figuring out how to
use it.

Don't miss the methods on Scanner: fetchColumnFamily(Text), fetchColumn(Text, Text), and fetchColumn(Column).
These are how you can easily do column family or column family + qualifier filtering.

For example, if you wanted to filter on the column family "foo" and the column qualifier "bar":

Scanner scan = connector.createScanner("table", auths);
scan.fetchColumn(new Text("foo"), new Text("bar"));
RowIterator rowIterator = new RowIterator(scan);
while (...) { ... }

The fetch*() methods are also accumulative. If you want to fetch multiple cf's (or cf+cq pairs),
you can invoke the method multiple times.

Ben Craig wrote:

Hey Guys I'm new to Accumulo and trying to learn how to query data.  I
think I've got the basics down like:

//create a scanner
Scanner scan = connector.createScanner( "table", auths );

//create a filter
IteratorSetting itr1 = new IteratorSetting( 1, "TimeFilter",
AgeOffFilter.class );
itr1.addOption( TTL, Long.toString( DEFAULT_QUERY_TIME ) );
scan.addScanIterator( itr1 );

//iterate over the resulting rows
RowIterator rowIterator = new RowIterator( scan );
while ( rowIterator.hasNext() )

I've been playing around with some of the built in filters and have been
able to apply multiple filters on top of each other.  Some of the
filters I'm having issues with where they take a complex java object and
not just option<String,String>

For example

When we use Iterator Settings the class is implicitly created but if I
want to use the ColumnQualifierFilter I need to create one and pass it a
set of columns.  I've been playing around with it for a while and havn't
been able to learn how to use it properly.

The constructor takes a sorted key value iterator.  How do I get this
sorted key value iterator?  Do I start with a scanner or do you start
with another type of scanner?  Do I just make one?
new ArrayList<SortedKeyValueIterator<Key,Value>>(); ? And the data goes
into it?

I've read through this Accumulo
<> book but it just
shows how you can use the Scanner/Iterator Settings to query.

If anyone has any suggestions / documentation / examples it be much




View raw message