hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Sorting columns
Date Sat, 19 Jun 2010 06:23:13 GMT
On Thu, Jun 17, 2010 at 12:18 PM, Andrey Stepachev <octo47@gmail.com> wrote:
> As i see in sources there no place, where kv sorted (except client
> Result.sorted() method). So we can get keyvalues from store and from
> memstore (and in this case we can get 1 3 5 from stores and 4 from memstore)
> in incorrect order.
>
> Or I miss something?
>

Data is sorted in hbase.  Scanning, we'll be running a scanner against
each data store element -- memstore and one for each store file -- and
we'll pop off the elements in order.  Thats the general story.  There
may once have been a legitimate reason for the client-side sort --
perhaps when our Get and Scan code paths differed it was needed -- but
as to whether it still required, I'm not sure.  I'd have to dig.  Any
one else?

>
>> > The rest of the data needs to be accessed occasionally. We want to avoid
>> > getting it shipped to the client as it makes our map reduce job go out of
>> > memory.
>> >
>>
>> You are not using incremental get on a row?  You should be able to get
>> your big rows piecemeal.
>>
> This scanner api changes was not included in 0.20.4 :( (infra row scanner).
>

Oh.

Sorry about that Andrey.  Somehow we missed your backport of
HBASE-1537.  I just applied it.  It'll appear in the 0.20.5RC4 I'm
rolling now.  Please excuse our bungling.

Yours,
St.Ack

Mime
View raw message