hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Can there be a doMiniBatchDelete in HRegion?
Date Mon, 25 Jun 2012 13:56:46 GMT
After testing in the cluster, please open a JIRA and attach result there.

Thanks for your effort, Anoop.

On Mon, Jun 25, 2012 at 6:50 AM, Anoop Sam John <anoopsj@huawei.com> wrote:

> I have made the new miniBatchDelete () and made the
> HTable#delete(List<Delete>) to call this new batch delete.
> Just tested initially with the one node cluster.  In that itself I am
> getting a performance boost which is very much promising.
> Only one CF and qualifier.
> 10K total rows delete with a batch of 100 deletes. Only deletes happening
> on the table from one thread.
> With the new way the net time taken is reduced by more than 1/10
> Will test in a 4 node cluster also. I think it will worth doing this
> change.
>
> -Anoop-
> ________________________________________
> From: yuzhihong@gmail.com [yuzhihong@gmail.com]
> Sent: Wednesday, June 20, 2012 6:31 PM
> To: dev@hbase.apache.org
> Cc: dev@hbase.apache.org
> Subject: Re: Can there be a doMiniBatchDelete in HRegion?
>
> I think you can issue large number of deletes on the same region and
> observe whether the proposed new method gives us performance boost.
>
> Thanks
>
>
>
> On Jun 20, 2012, at 2:49 AM, Anoop Sam John <anoopsj@huawei.com> wrote:
>
> > Hi Devs
> >
> >              There is a batch put support in the HRegion level. When the
> put(List<Put>) happens from client, Puts corresponding to one region might
> get grouped together and handled as a batch.[Depending on the availability
> of rowlocks..   code in HRegion#doMiniBatchPut] For this batch there will
> be single write and sync into the HLog file.
> >
> >
> >
> > A similar kind of delete operation, I am not able to see in HRegion. The
> HTable#delete(List<Delete>) groups the Deletes for the same RS and make one
> n/w call only. But within the RS, there will be N number of delete calls on
> the region one by one. This will include N number of HLog write and sync.
> If this also can be grouped can we get better performance for the multi row
> delete.  Is there any problem in doing this batch delete? I am not sure any
> JIRA is already present for this.
> >
> >
> >
> > Note : Hregion#mutateRowsWithLock().. we do batch operations of Puts and
> Deletes(also)
> >
> >
> >
> > -Anoop-
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message