accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Craig <ben.cra...@gmail.com>
Subject Re: Querying Accumulo Data Using Java
Date Thu, 28 Apr 2016 17:35:24 GMT
Awesome. Thanks guys.

On Thu, Apr 28, 2016 at 2:10 PM, Josh Elser <josh.elser@gmail.com> wrote:

> A common pattern we have (but by no means the only tool) are static
> methods on the Iterator/Filter to help serialize domain-specific
> configuration in string=>string format.
>
> e.g. LongCombiner.setEncodingType(IteratorSetting, LongCombiner.Type)
>
> Dan Blum wrote:
>
>> You can’t pass objects because the iterator stack will be run in a
>> different JVM; anything that isn’t a string or coercible to a string
>> would need to be serialized. So if you have something complex you will
>> need to serialize it into a string somehow and then deserialize it in
>> your filter class, which is quite doable.
>>
>> *From:*Ben Craig [mailto:ben.craig2@gmail.com]
>> *Sent:* Thursday, April 28, 2016 12:54 PM
>> *To:* user@accumulo.apache.org
>> *Subject:* Re: Querying Accumulo Data Using Java
>>
>> Hey Josh,
>>
>>
>> Thanks for the response. I was getting pretty lost trying to use the
>> Internal Iterator. I will try using the fetchColumn on the scanner. I
>> guess I have one last question is there any possible way to pass a java
>> object to a custom filter or are we limited to the PropertyMap of
>> <String,String> ? I think its the String,String will work in almost all
>> cases I need to do but am more curious than anything.
>>
>> Thanks,
>>
>>
>> Ben
>>
>> On Thu, Apr 28, 2016 at 1:34 PM, Josh Elser <josh.elser@gmail.com
>> <mailto:josh.elser@gmail.com>> wrote:
>>
>> Hi Ben,
>>
>> Looks like you're on the right track. Iterator priorities are a little
>> obtuse at first glance; you probably want to change the 1 to 15 (we can
>> touch on the "why" later).
>>
>> As far as Iterators/Filters that you should feel comfortable using,
>> check out:
>>
>> https://github.com/apache/accumulo/tree/master/core/src/main/java/org/apache/accumulo/core/iterators/user
>>
>> The "system" package are not meant for public consumption (and often are
>> used to implement internal functionality). This is probably why you're
>> having a hard time figuring out how to use it.
>>
>> Don't miss the methods on Scanner: fetchColumnFamily(Text),
>> fetchColumn(Text, Text), and fetchColumn(Column). These are how you can
>> easily do column family or column family + qualifier filtering.
>>
>> For example, if you wanted to filter on the column family "foo" and the
>> column qualifier "bar":
>>
>> ```
>> Scanner scan = connector.createScanner("table", auths);
>> scan.fetchColumn(new Text("foo"), new Text("bar"));
>> RowIterator rowIterator = new RowIterator(scan);
>> while (...) { ... }
>> ```
>>
>> The fetch*() methods are also accumulative. If you want to fetch
>> multiple cf's (or cf+cq pairs), you can invoke the method multiple times.
>>
>> Ben Craig wrote:
>>
>> Hey Guys I'm new to Accumulo and trying to learn how to query data. I
>> think I've got the basics down like:
>>
>> //create a scanner
>> Scanner scan = connector.createScanner( "table", auths );
>>
>> //create a filter
>> IteratorSetting itr1 = new IteratorSetting( 1, "TimeFilter",
>> AgeOffFilter.class );
>> itr1.addOption( TTL, Long.toString( DEFAULT_QUERY_TIME ) );
>> scan.addScanIterator( itr1 );
>>
>> //iterate over the resulting rows
>> RowIterator rowIterator = new RowIterator( scan );
>> while ( rowIterator.hasNext() )
>> {
>> }
>>
>> I've been playing around with some of the built in filters and have been
>> able to apply multiple filters on top of each other. Some of the
>> filters I'm having issues with where they take a complex java object and
>> not just option<String,String>
>>
>> For example ColumnQualifierFilter.java
>> <
>> https://github.com/apache/accumulo/blob/master/core/src/main/java/org/apache/accumulo/core/iterators/system/ColumnQualifierFilter.java
>> >
>>
>> When we use Iterator Settings the class is implicitly created but if I
>> want to use the ColumnQualifierFilter I need to create one and pass it a
>> set of columns. I've been playing around with it for a while and havn't
>> been able to learn how to use it properly.
>>
>> The constructor takes a sorted key value iterator. How do I get this
>> sorted key value iterator? Do I start with a scanner or do you start
>> with another type of scanner? Do I just make one?
>> new ArrayList<SortedKeyValueIterator<Key,Value>>(); ? And the data goes
>> into it?
>>
>>
>>
>> I've read through this Accumulo
>> <http://shop.oreilly.com/product/0636920032304.do> book but it just
>> shows how you can use the Scanner/Iterator Settings to query.
>>
>> If anyone has any suggestions / documentation / examples it be much
>> appreciated.
>>
>> Thanks,
>>
>> Ben
>>
>>

Mime
View raw message