hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars George <l...@worldlingo.com>
Subject Re: Use Delete with write-buffer in HTable
Date Tue, 29 Sep 2009 16:33:16 GMT
Hey Michael,

Sure it would change to take Writable's, but is that a big deal? The 
problem is that the batch delete is an explicit call. But using TOF does 
not make use of it because it relies on the table to commit and flush. 
Hence my suggestion to do the same for Delete's - especially seeing how 
slow single deletes are when doing thousands of them.

You did not say hey or ney yet if I read correctly?

Lars

stack wrote:
> The write buffer looks like this currently:
>
>   private final ArrayList<Put> writeBuffer = new ArrayList<Put>();
>
> ... so that would have to change.
>
> A batch delete was added to TRUNK and to head of the 0.20 branch recently
> FYI.
>
> HBASE-1845 is about cleaning up our batching and adding batch Get to the
> mix.
>
> St.Ack
>
> On Tue, Sep 29, 2009 at 3:36 AM, Lars George <lars@worldlingo.com> wrote:
>
>   
>> Hi Guys,
>>
>> Is there a reason why Delete's are not also put on the write buffer like
>> Put's and then flushed out. That way we get an implicit batch delete (using
>> the new batch delete calls internally, just like the batch put does) and MR
>> jobs that output Put or Delete are automatically using the buffer for much
>> better performance.
>>
>> I recently had an MR with 18K deletes and since batch delete was not yet
>> available it crashed my job constantly because of the huge amount of RPC
>> calls.
>>
>> Opinions?
>>
>> Lars
>>
>>     
>
>   

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message