accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: LocalityGroupDeleter
Date Sun, 17 Jul 2016 14:58:47 GMT
Neat stuff, Russ.

Now that I'm thinking about it more, I'm kind of surprised this isn't 
something that is just auto-magically fast and provided for you. The 
notion of a locality group makes it seem logical that it would just be 
able to prune that "segment" easily. Not so much in practice, I guess!

Any interest getting a Java-variant of your LocalityGroupDeleter 
committed to the codebase and write up some docs in the user manual? I 
think this would useful for others in the future. HTH too.

- Josh

Russ Weeks wrote:
> Hey, folks,
>
> Yesterday I asked on IRC about whether there was an optimized way to delete
> all key/value pairs in a locality group. Keith had the great suggestion
> that, since compactions are done separately for each locality group, I
> could write a custom iterator that returned false from hasTop if the
> iterator was seeked with the right column families.
>
> If it's useful to anyone else, the code is in this gist:
> https://gist.github.com/rweeks/bedb96e20fcb8d2d2f27c555494181c5 and it
> works pretty well, thanks Keith!
>
> It would be more convenient for the end-user to configure the iterator with
> a specific locality group, instead of a set of column families that must
> match a locality group for the iterator to work correctly. Unfortunately
> the iterator doesn't have the necessary context to map a locality group
> name to the CFs in that group (as far as I can tell).
>
> -Russ
>

Mime
View raw message