incubator-accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bil...@apache.org
Subject svn commit: r1228242 - /incubator/accumulo/branches/1.4/docs/examples/README.filter
Date Fri, 06 Jan 2012 16:01:35 GMT
Author: billie
Date: Fri Jan  6 16:01:35 2012
New Revision: 1228242

URL: http://svn.apache.org/viewvc?rev=1228242&view=rev
Log:
ACCUMULO-275 rewrote README.filter

Modified:
    incubator/accumulo/branches/1.4/docs/examples/README.filter

Modified: incubator/accumulo/branches/1.4/docs/examples/README.filter
URL: http://svn.apache.org/viewvc/incubator/accumulo/branches/1.4/docs/examples/README.filter?rev=1228242&r1=1228241&r2=1228242&view=diff
==============================================================================
--- incubator/accumulo/branches/1.4/docs/examples/README.filter (original)
+++ incubator/accumulo/branches/1.4/docs/examples/README.filter Fri Jan  6 16:01:35 2012
@@ -17,32 +17,31 @@ Notice:    Licensed to the Apache Softwa
            under the License.
 
 This is a simple filter example.  It uses the AgeOffFilter that is provided as 
-part of the core package org.apache.accumulo.core.iterators.filter.  Filters are used by
-the FilteringIterator to select desired key/value pairs (or weed out undesired 
-ones).  Filters implement the org.apache.accumulo.core.iterators.iterators.filter.Filter
interface which 
-contains a method accept(Key k, Value v).  This method returns true if the key, 
-value pair are to be delivered and false if they are to be ignored.
+part of the core package org.apache.accumulo.core.iterators.user.  Filters are 
+iterators that select desired key/value pairs (or weed out undesired ones).  
+Filters extend the org.apache.accumulo.core.iterators.iterators.Filter class 
+and must implement a method accept(Key k, Value v).  This method returns true 
+if the key/value pair are to be delivered and false if they are to be ignored.
+Filter takes a "negate" parameter which defaults to false.  If set to true, the
+return value of the accept method is negated, so that key/value pairs accepted
+by the method are omitted by the Filter.
 
     username@instance> createtable filtertest
-    username@instance filtertest> setiter -t filtertest -scan -p 10 -n myfilter -filter
-    FilteringIterator uses Filters to accept or reject key/value pairs
-    ----------> entering options: <filterPriorityNumber> <ageoff|regex|filterClass>
-    ----------> set org.apache.accumulo.core.iterators.FilteringIterator option (<name>
<value>, hit enter to skip): 0 ageoff
-    ----------> set org.apache.accumulo.core.iterators.FilteringIterator option (<name>
<value>, hit enter to skip): 
+    username@instance filtertest> setiter -t filtertest -scan -p 10 -n myfilter -ageoff
     AgeOffFilter removes entries with timestamps more than <ttl> milliseconds old
-    ----------> set org.apache.accumulo.core.iterators.filter.AgeOffFilter parameter currentTime,
if set, use the given value as the absolute time in milliseconds as the current time of day:

-    ----------> set org.apache.accumulo.core.iterators.filter.AgeOffFilter parameter ttl,
time to live (milliseconds): 30000
-    username@instance filtertest> 
-    
+    ----------> set AgeOffFilter parameter negate, default false keeps k/v that pass accept
method, true rejects k/v that pass accept method: 
+    ----------> set AgeOffFilter parameter ttl, time to live (milliseconds): 30000
+    ----------> set AgeOffFilter parameter currentTime, if set, use the given value as
the absolute time in milliseconds as the current time of day: 
     username@instance filtertest> scan
     username@instance filtertest> insert foo a b c
     username@instance filtertest> scan
-    foo a:b []	c
+    foo a:b []    c
+    username@instance filtertest> 
     
 ... wait 30 seconds ...
     
     username@instance filtertest> scan
-    username@instance filtertest>
+    username@instance filtertest> 
 
 Note the absence of the entry inserted more than 30 seconds ago.  Since the
 scope was set to "scan", this means the entry is still in Accumulo, but is
@@ -50,56 +49,62 @@ being filtered out at query time.  To de
 the ages of their timestamps, AgeOffFilters should be set up for the "minc"
 and "majc" scopes, as well.
 
-To force an ageoff in the persisted data, after setting up the ageoff iterator 
+To force an ageoff of the persisted data, after setting up the ageoff iterator 
 on the "minc" and "majc" scopes you can flush and compact your table. This will
 happen automatically as a background operation on any table that is being 
-actively written to, but these are the commands to force compaction:
+actively written to, but can also be requested in the shell.
 
-    username@instance filtertest> setiter -t filtertest -scan -minc -majc -p 10 -n myfilter
-filter
-    FilteringIterator uses Filters to accept or reject key/value pairs
-    ----------> entering options: <filterPriorityNumber> <ageoff|regex|filterClass>
-    ----------> set org.apache.accumulo.core.iterators.FilteringIterator option (<name>
<value>, hit enter to skip): 0 ageoff
-    ----------> set org.apache.accumulo.core.iterators.FilteringIterator option (<name>
<value>, hit enter to skip): 
+The first setiter command used the special -ageoff flag to specify the 
+AgeOffFilter, but any Filter can be configured by using the -class flag.  The 
+following commands show how to enable the AgeOffFilter for the minc and majc
+scopes using the -class flag, then flush and compact the table.
+
+    username@instance filtertest> setiter -t filtertest -minc -majc -p 10 -n myfilter
-class org.apache.accumulo.core.iterators.user.AgeOffFilter
     AgeOffFilter removes entries with timestamps more than <ttl> milliseconds old
-    ----------> set org.apache.accumulo.core.iterators.filter.AgeOffFilter parameter currentTime,
if set, use the given value as the absolute time in milliseconds as the current time of day:

-    ----------> set org.apache.accumulo.core.iterators.filter.AgeOffFilter parameter ttl,
time to live (milliseconds): 30000
-    username@instance filtertest> 
-    
-    username@instance filtertest> flush -t filtertest
-    08 11:13:55,745 [shell.Shell] INFO : Flush of table filtertest initiated...
-    username@instance filtertest> compact -t filtertest
-    08 11:14:10,800 [shell.Shell] INFO : Compaction of table filtertest scheduled for 20110208111410EST
-    username@instance filtertest> 
+    ----------> set AgeOffFilter parameter negate, default false keeps k/v that pass accept
method, true rejects k/v that pass accept method: 
+    ----------> set AgeOffFilter parameter ttl, time to live (milliseconds): 30000
+    ----------> set AgeOffFilter parameter currentTime, if set, use the given value as
the absolute time in milliseconds as the current time of day: 
+    username@instance filtertest> flush
+    06 10:42:24,806 [shell.Shell] INFO : Flush of table filtertest initiated...
+    username@instance filtertest> compact
+    06 10:42:36,781 [shell.Shell] INFO : Compaction of table filtertest started for given
range
+    username@instance filtertest> flush -t filtertest -w
+    06 10:42:52,881 [shell.Shell] INFO : Flush of table filtertest completed.
+    username@instance filtertest> compact -t filtertest -w
+    06 10:43:00,632 [shell.Shell] INFO : Compacting table ...
+    06 10:43:01,307 [shell.Shell] INFO : Compaction of table filtertest completed for given
range
+    username@instance filtertest>
 
-After the compaction runs, the newly created files will not contain any data that should
be aged off, and the
-Accumulo garbage collector will remove the old files.
+By default, flush and compact execute in the background, but with the -w flag
+they will wait to return until the operation has completed.  Both are 
+demonstrated above, though only one call to each would be necessary.  A 
+specific table can be specified with -t.
+
+After the compaction runs, the newly created files will not contain any data 
+that should have been aged off, and the Accumulo garbage collector will remove 
+the old files.
 
-To see the iterator settings for a table, use:
+To see the iterator settings for a table, use config.
 
     username@instance filtertest> config -t filtertest -f iterator
-    ---------+------------------------------------------+----------------------------------------------------------
-    SCOPE    | NAME                                     | VALUE
-    ---------+------------------------------------------+----------------------------------------------------------
-    table    | table.iterator.majc.myfilter .............. | 10,org.apache.accumulo.core.iterators.FilteringIterator
-    table    | table.iterator.majc.myfilter.opt.0 ........ | org.apache.accumulo.core.iterators.filter.AgeOffFilter
-    table    | table.iterator.majc.myfilter.opt.0.ttl .... | 30000
-    table    | table.iterator.majc.vers .................. | 20,org.apache.accumulo.core.iterators.VersioningIterator
+    ---------+---------------------------------------------+---------------------------------------------------------------------------
+    SCOPE    | NAME                                        | VALUE
+    ---------+---------------------------------------------+---------------------------------------------------------------------------
+    table    | table.iterator.majc.myfilter .............. | 10,org.apache.accumulo.core.iterators.user.AgeOffFilter
+    table    | table.iterator.majc.myfilter.opt.ttl ...... | 30000
+    table    | table.iterator.majc.vers .................. | 20,org.apache.accumulo.core.iterators.user.VersioningIterator
     table    | table.iterator.majc.vers.opt.maxVersions .. | 1
-    table    | table.iterator.minc.myfilter .............. | 10,org.apache.accumulo.core.iterators.FilteringIterator
-    table    | table.iterator.minc.myfilter.opt.0 ........ | org.apache.accumulo.core.iterators.filter.AgeOffFilter
-    table    | table.iterator.minc.myfilter.opt.0.ttl .... | 30000
-    table    | table.iterator.minc.vers .................. | 20,org.apache.accumulo.core.iterators.VersioningIterator
+    table    | table.iterator.minc.myfilter .............. | 10,org.apache.accumulo.core.iterators.user.AgeOffFilter
+    table    | table.iterator.minc.myfilter.opt.ttl ...... | 30000
+    table    | table.iterator.minc.vers .................. | 20,org.apache.accumulo.core.iterators.user.VersioningIterator
     table    | table.iterator.minc.vers.opt.maxVersions .. | 1
-    table    | table.iterator.scan.myfilter .............. | 10,org.apache.accumulo.core.iterators.FilteringIterator
-    table    | table.iterator.scan.myfilter.opt.0 ........ | org.apache.accumulo.core.iterators.filter.AgeOffFilter
-    table    | table.iterator.scan.myfilter.opt.0.ttl .... | 30000
-    table    | table.iterator.scan.vers .................. | 20,org.apache.accumulo.core.iterators.VersioningIterator
+    table    | table.iterator.scan.myfilter .............. | 10,org.apache.accumulo.core.iterators.user.AgeOffFilter
+    table    | table.iterator.scan.myfilter.opt.ttl ...... | 30000
+    table    | table.iterator.scan.vers .................. | 20,org.apache.accumulo.core.iterators.user.VersioningIterator
     table    | table.iterator.scan.vers.opt.maxVersions .. | 1
-    ---------+------------------------------------------+----------------------------------------------------------
+    ---------+---------------------------------------------+---------------------------------------------------------------------------
     username@instance filtertest> 
 
-If you would like to apply multiple filters, this can be done using a single
-iterator. Just continue adding entries during the 
-"set org.apache.accumulo.core.iterators.FilteringIterator option" step.
-Make sure to order the filterPriorityNumbers in the order you would like
-the filters to be applied.
+When setting new iterators, make sure to order their priority numbers 
+(specified with -p) in the order you would like the iterators to be applied.
+Also, each iterator must have a unique name and priority within each scope.



Mime
View raw message