accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Querying Accumulo Data Using Java
Date Thu, 28 Apr 2016 17:10:02 GMT
A common pattern we have (but by no means the only tool) are static 
methods on the Iterator/Filter to help serialize domain-specific 
configuration in string=>string format.

e.g. LongCombiner.setEncodingType(IteratorSetting, LongCombiner.Type)

Dan Blum wrote:
> You can’t pass objects because the iterator stack will be run in a
> different JVM; anything that isn’t a string or coercible to a string
> would need to be serialized. So if you have something complex you will
> need to serialize it into a string somehow and then deserialize it in
> your filter class, which is quite doable.
>
> *From:*Ben Craig [mailto:ben.craig2@gmail.com]
> *Sent:* Thursday, April 28, 2016 12:54 PM
> *To:* user@accumulo.apache.org
> *Subject:* Re: Querying Accumulo Data Using Java
>
> Hey Josh,
>
>
> Thanks for the response. I was getting pretty lost trying to use the
> Internal Iterator. I will try using the fetchColumn on the scanner. I
> guess I have one last question is there any possible way to pass a java
> object to a custom filter or are we limited to the PropertyMap of
> <String,String> ? I think its the String,String will work in almost all
> cases I need to do but am more curious than anything.
>
> Thanks,
>
>
> Ben
>
> On Thu, Apr 28, 2016 at 1:34 PM, Josh Elser <josh.elser@gmail.com
> <mailto:josh.elser@gmail.com>> wrote:
>
> Hi Ben,
>
> Looks like you're on the right track. Iterator priorities are a little
> obtuse at first glance; you probably want to change the 1 to 15 (we can
> touch on the "why" later).
>
> As far as Iterators/Filters that you should feel comfortable using,
> check out:
> https://github.com/apache/accumulo/tree/master/core/src/main/java/org/apache/accumulo/core/iterators/user
>
> The "system" package are not meant for public consumption (and often are
> used to implement internal functionality). This is probably why you're
> having a hard time figuring out how to use it.
>
> Don't miss the methods on Scanner: fetchColumnFamily(Text),
> fetchColumn(Text, Text), and fetchColumn(Column). These are how you can
> easily do column family or column family + qualifier filtering.
>
> For example, if you wanted to filter on the column family "foo" and the
> column qualifier "bar":
>
> ```
> Scanner scan = connector.createScanner("table", auths);
> scan.fetchColumn(new Text("foo"), new Text("bar"));
> RowIterator rowIterator = new RowIterator(scan);
> while (...) { ... }
> ```
>
> The fetch*() methods are also accumulative. If you want to fetch
> multiple cf's (or cf+cq pairs), you can invoke the method multiple times.
>
> Ben Craig wrote:
>
> Hey Guys I'm new to Accumulo and trying to learn how to query data. I
> think I've got the basics down like:
>
> //create a scanner
> Scanner scan = connector.createScanner( "table", auths );
>
> //create a filter
> IteratorSetting itr1 = new IteratorSetting( 1, "TimeFilter",
> AgeOffFilter.class );
> itr1.addOption( TTL, Long.toString( DEFAULT_QUERY_TIME ) );
> scan.addScanIterator( itr1 );
>
> //iterate over the resulting rows
> RowIterator rowIterator = new RowIterator( scan );
> while ( rowIterator.hasNext() )
> {
> }
>
> I've been playing around with some of the built in filters and have been
> able to apply multiple filters on top of each other. Some of the
> filters I'm having issues with where they take a complex java object and
> not just option<String,String>
>
> For example ColumnQualifierFilter.java
> <https://github.com/apache/accumulo/blob/master/core/src/main/java/org/apache/accumulo/core/iterators/system/ColumnQualifierFilter.java>
>
> When we use Iterator Settings the class is implicitly created but if I
> want to use the ColumnQualifierFilter I need to create one and pass it a
> set of columns. I've been playing around with it for a while and havn't
> been able to learn how to use it properly.
>
> The constructor takes a sorted key value iterator. How do I get this
> sorted key value iterator? Do I start with a scanner or do you start
> with another type of scanner? Do I just make one?
> new ArrayList<SortedKeyValueIterator<Key,Value>>(); ? And the data goes
> into it?
>
>
>
> I've read through this Accumulo
> <http://shop.oreilly.com/product/0636920032304.do> book but it just
> shows how you can use the Scanner/Iterator Settings to query.
>
> If anyone has any suggestions / documentation / examples it be much
> appreciated.
>
> Thanks,
>
> Ben
>

Mime
View raw message