incubator-accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Billie J Rinaldi <billie.j.rina...@ugov.gov>
Subject Re: Filter use
Date Fri, 16 Dec 2011 14:47:14 GMT
You should place it in the accumulo/lib/ext directory.  The classloader will pick up changes
you make to jars in that directory.

Billie


----- Original Message -----
> From: "Joey Daughtery" <jdaughtery@t-sciences.com>
> To: accumulo-user@incubator.apache.org
> Sent: Friday, December 16, 2011 9:26:19 AM
> Subject: Re: Filter use
> I think I have found the issue. It is throwing a class not found
> exception on my subclass. I did not add it to the accumulo build. I
> will do that and try again.
> 
> I am guessing that I should be able to create a client jar with my
> subclass and place it into the accumulo/lib directory.
> 
> 
> Thanks
> 
> Joe
> 
> 
> On Fri, Dec 16, 2011 at 8:52 AM, Eric Newton < eric.newton@gmail.com >
> wrote:
> 
> 
> Do you see a stack trace in the tablet server log?
> 
> 
> 
> 
> 
> On Fri, Dec 16, 2011 at 8:47 AM, Joey Daughtery <
> jdaughtery@t-sciences.com > wrote:
> 
> 
> All
> I was finally able to get back to this. When I implemented the
> following on Accumulo 1.4 and execute, I get the exception below.
> 
> Also, where are the option keys defined for IteratorSetting.addOption?
> 
> 1. I subclassed the org.apache.accumulo.core.iterators.Filter object
> and implemented the accept(Key key, Value value) method.
> 
> 2. Then I utilized it in the following way:
> IteratorSetting is = new IteratorSetting(15, "idFilter",
> fi.getClass());
> is.addOption("valuesToSkip", userId);
> 
> 
> Range r = new Range();
> Scanner s = conn.createScanner("index_table", scanauths);
> s.setRange(r);
> s.addScanIterator(is);
> 
> 
> 
> Caused by: org.apache.thrift.TApplicationException: Internal error
> processing startScan
> at
> org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
> at
> org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.recv_startScan(TabletClientService.java:184)
> at
> org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Client.startScan(TabletClientService.java:157)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at cloudtrace.instrument.thrift.TraceWrap$2.invoke(TraceWrap.java:67)
> at $Proxy1.startScan(Unknown Source)
> at
> org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:415)
> at
> org.apache.accumulo.core.client.impl.ThriftScanner.scan(ThriftScanner.java:295)
> ... 4 more
> 
> Thanks
> 
> Joe
> 
> 
> 
> 
> On Mon, Dec 12, 2011 at 10:23 AM, Billie J Rinaldi <
> billie.j.rinaldi@ugov.gov > wrote:
> 
> 
> 
> On Date: Dec 12, 2011 9:29 AM, "Aaron Cordova" <
> aaron.cordova@interllective.com > wrote:
> > 3. Configure your table to use your filtering iterator as described
> > in
> > this section of the user manual:
> > http://incubator.apache.org/accumulo/user_manual_1.3-incubating/Table_Configuration.html
> > , where you specify the name of your Java class as the first option
> > in
> > the Shell, rather than 'ageoff' or 'regex'.
> 
> Unfortunately the manual does not say how to specify filters
> programmatically. This is somewhat confusing in 1.3 because the Filter
> class you will be writing is a plugin to the FilteringIterator, and
> the Filter is not itself an Iterator. Configuring the filter in the
> shell and just setting its parameters programmatically is a
> possibility, but the syntax Aaron gave for setting the parameter is
> incorrect. Here is an example of configuring a FilteringIterator:
> 
> scanner.setScanIterators(priority,
> "org.apache.accumulo.core.iterators.FilteringIterator", "myFilter");
> scanner.setScanIteratorOption("myFilter", "0",
> "org.apache.accumulo.core.iterators.filter.MyFilter");
> scanner.setScanIteratorOption("myFilter", "0.valuesToSkip", "10");
> scanner.setScanIteratorOption("myFilter", "1",
> "org.apache.accumulo.core.iterators.filter.AgeOffFilter");
> scanner.setScanIteratorOption("myFilter", "1.ttl", ttl);
> 
> If you can start coding to 1.4 instead of 1.3, I would highly
> recommend it. All provided user iterators in 1.4 are located in the
> org.apache.accumulo.core.iterators.user package. Filters are Iterators
> and are much easier to configure:
> 
> IteratorSetting setting = new IteratorSetting(priority,
> MyFilter.class);
> setting.addOption("valuesToSkip", "10");
> scanner.addScanIterator(setting);
> 
> ** it is recommended to configure the IteratorSetting with static
> methods instead of addOption,
> e.g. TimestampFilter.setRange(setting, "20120101000000GMT",
> "20120102000000GMT");
> 
> One of the above options should work. However, filtering based on the
> Value, the unsorted portion of a sorted key/value store, is not going
> to be an efficient operation. Your query will have to scan over all
> the entries in the table to find the Values you want. This might be ok
> if the query returns most of the data in the table, but if it just
> returns a small fraction of the data it would be better to figure out
> how to get the sorting to work to your advantage.
> 
> Billie

Mime
View raw message