accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christopher <ctubb...@apache.org>
Subject Re: AgeOff Filter - Compaction needed?
Date Tue, 26 Nov 2013 18:46:03 GMT
Compactions re-write data. Iterators simply transform or filter data.
Configuring an iterator, such as the AgeOff iterator to a compaction
scope, means that data will be removed as compactions occur. You don't
need to force a compaction, because they happen over time, but you
might want to, to ensure the data is aged-off right away. It's
possible that an idle tablet will never compact again, and run your
age-off iterators. For that, you will need to force a compaction.

You may also want to apply your age off iterator to all scopes (minc,
majc, and scan), so that you have a consistent view in your scans
until things are permanently re-written without the old data (which
happens at the majc scope).

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii


On Fri, Nov 22, 2013 at 10:58 AM, Slater, David M.
<David.Slater@jhuapl.edu> wrote:
> Hi, I would like to automatically ageoff old data; however my understanding
> of AgeOff iterators is that they would require a compaction or scan to take
> effect. For information that is in tablets that are no longer being added to
> or scanned, the ageoff filter would not take effect, correct?
>
>
>
> Would I need to force a compaction or merge in order to remove those
> records, or would it be better to write my own script to remove rows? (I’m
> using 1.4 – is this behavior different in 1.5 or 1.6?)
>
>
>
> Thanks,
> David

Mime
View raw message