hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Can there be a doMiniBatchDelete in HRegion?
Date Mon, 25 Jun 2012 16:11:50 GMT
>From another thread, the following is related to the optimization Anoop is
testing:

In HRegionServer:
  public <R> MultiResponse multi(MultiAction<R> multi) throws IOException {
...
      for (Action<R> a : actionsForRegion) {
        action = a.getAction();
...
          if (action instanceof Delete) {
            delete(regionName, (Delete) action);

I think if we group the deletes of actionsForRegion, we can utilize the
following:
  public int delete(final byte[] regionName, final List<Delete> deletes)

On Mon, Jun 25, 2012 at 6:56 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> After testing in the cluster, please open a JIRA and attach result there.
>
> Thanks for your effort, Anoop.
>
>
> On Mon, Jun 25, 2012 at 6:50 AM, Anoop Sam John <anoopsj@huawei.com>wrote:
>
>> I have made the new miniBatchDelete () and made the
>> HTable#delete(List<Delete>) to call this new batch delete.
>> Just tested initially with the one node cluster.  In that itself I am
>> getting a performance boost which is very much promising.
>> Only one CF and qualifier.
>> 10K total rows delete with a batch of 100 deletes. Only deletes happening
>> on the table from one thread.
>> With the new way the net time taken is reduced by more than 1/10
>> Will test in a 4 node cluster also. I think it will worth doing this
>> change.
>>
>> -Anoop-
>> ________________________________________
>> From: yuzhihong@gmail.com [yuzhihong@gmail.com]
>> Sent: Wednesday, June 20, 2012 6:31 PM
>> To: dev@hbase.apache.org
>> Cc: dev@hbase.apache.org
>> Subject: Re: Can there be a doMiniBatchDelete in HRegion?
>>
>> I think you can issue large number of deletes on the same region and
>> observe whether the proposed new method gives us performance boost.
>>
>> Thanks
>>
>>
>>
>> On Jun 20, 2012, at 2:49 AM, Anoop Sam John <anoopsj@huawei.com> wrote:
>>
>> > Hi Devs
>> >
>> >              There is a batch put support in the HRegion level. When
>> the put(List<Put>) happens from client, Puts corresponding to one region
>> might get grouped together and handled as a batch.[Depending on the
>> availability of rowlocks..   code in HRegion#doMiniBatchPut] For this batch
>> there will be single write and sync into the HLog file.
>> >
>> >
>> >
>> > A similar kind of delete operation, I am not able to see in HRegion.
>> The HTable#delete(List<Delete>) groups the Deletes for the same RS and make
>> one n/w call only. But within the RS, there will be N number of delete
>> calls on the region one by one. This will include N number of HLog write
>> and sync. If this also can be grouped can we get better performance for the
>> multi row delete.  Is there any problem in doing this batch delete? I am
>> not sure any JIRA is already present for this.
>> >
>> >
>> >
>> > Note : Hregion#mutateRowsWithLock().. we do batch operations of Puts
>> and Deletes(also)
>> >
>> >
>> >
>> > -Anoop-
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message