accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Turner <ke...@deenlo.com>
Subject Re: why not check TTL interval
Date Sat, 06 Jun 2015 16:03:18 GMT
I was thinking about how to generalize this in 1266[1]

[1]: https://issues.apache.org/jira/browse/ACCUMULO-1266

Sent from phone. Please excuse typos and brevity.
On Jun 6, 2015 1:50 AM, "Lu Qin" <luq.java@gmail.com> wrote:

> Accumulo do minor-compaction and major-compaction depends on a thresold
> value ,why not do age-off interval-automatic by ttl value.
> If I do it use crontab,when I add a new table ,I must update my crontab
>
>
> > 在 2015年6月6日,13:37,Josh Elser <josh.elser@gmail.com> 写道:
> >
> > The decrease in performance you see is probably because the iterator
> must read a significant amount of old data. If you don't write new data to
> a table, Accumulo will not run any compactions and no data will age-off in
> the files on HDFS.
> >
> > I think it would be fairly common to use crontab to regularly schedule
> compactions over your table so that data is automatically deleted (e.g.
> nightly). Accumulo doesn't contain any means to automate this internally.
> >
> > Lu Qin wrote:
> >> I have a big table about 38B entries, and I set a ageoff iterator with
> a ttl about 3 days,I set the iteratorPriority is 10 and apply it in
> all-scopes.
> >>
> >> I stop write data into it about one week,and now I scan it ,but it wait
> so long. I check the monitor page,it show me that the scan speed is 80w
> entries/s.
> >>
> >> I think the ageoff is a diferent iterator than others,if all data is
> out the ttl,when I scan the table,it will scan all data in the table and
> decide to remove it,right? Why not do this interval ?
> >>
> >> Thanks
>
>

Mime
View raw message