accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Medinets <david.medin...@gmail.com>
Subject Re: Pagination in Accumulo (also D4M Data Explorer!)
Date Fri, 16 May 2014 16:51:43 GMT
Josh, this morning I woke up and remembered that I wrote
http://affy.blogspot.com/2012/11/how-can-i-use-reverse-sort-on-generic.html
about 18 months ago. I can easily add a reverse index in order to
extend
the D4M schema.

I'm glad to see that reverse scanning is possible in HBase.


On Thu, May 15, 2014 at 8:43 PM, Josh Elser <josh.elser@gmail.com> wrote:

> Reverse scanning isn't necessarily infeasible: https://issues.apache.org/
> jira/browse/HBASE-4811
>
> This might be something cool that could be implemented to make this sort
> of thing easiser.
>
> The pagination isolation you mention in Approach B is interesting. I'm
> curious as to how clone'ing tables would work to get you this. I imagine
> for a highly active system (read and write) this would start to break down
> pretty quickly.
>
> I think having a "loose" interpretation of the actual page you're on would
> be easiest to implement without omitting data (your "next?"). This would be
> a bit easier with a reverse scanner. The "page" then becomes a bit of a
> guess -- really, it's just a guess at the section of records being viewed
> (e.g. 20% behind and 80% forward). That might make for a more honest
> pagination view instead of explicitly listing page numbers which you know
> will change.
>
> Have you considered creating a reverse row index to get around the lack of
> a reverse scanner?
>
>
> On 5/14/14, 3:19 PM, David Medinets wrote:
>
>> While working on the D4M Data Explorer
>> (https://github.com/medined/D4M_Schema) web application, I started
>> thinking about how to paginate. The results are at
>> https://github.com/medined/D4M_Schema/blob/master/docs/pagination.md.
>> I'll reproduce only the first paragraph below since I don't want you to
>> miss out on my cool images.
>>
>> - First Paragraph -
>> Pagination in Accumulo is not simple. Pages are not deterministic since
>> the data can be constantly changing. Also authorization levels can
>> change the number of returned results. Another consideration is that
>> Accumulo tables can only be scanned forwards and not backwards. With
>> these factors in mind, I am implementing the following technique. I hope
>> the community can point out flaws and provide improvements.
>> ---
>>
>> Several people helped to review the page via private emails. Thanks! If
>> you want credit, please step-up and self-identify.
>>
>> p.s. - The D4M Data Explorer is not too existing just yet. But it should
>> get better. If you're interested I'd be glad for any help.
>>
>>

Mime
View raw message