hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Biju N <bijuatapa...@gmail.com>
Subject Re: Asking about the trend of increasing latency, hbase spikes.
Date Thu, 08 Oct 2015 00:38:44 GMT
Looks like the attachments are not viewable.

On Wed, Oct 7, 2015 at 8:35 PM, UI-JIN LIM <ujlim@lgcns.com> wrote:

> Hi. This is Ui Jin, Lim in Korea, LG CNS
>
>
>
> We had setup and are operating hbase 0.98.13 on our customer, iptv
> Company.
>
> Our customer claim about this spike. Following is our status.
>
>
>
> . We had been operating with BlockCache(heap) for 3 months. The ratio of
> above 100ms has been reached 6.69% of total transaction.
>
>   The following graph show the trend of executions, above 100ms per
> minute, at June 17th. (This is the response time of 2~3 Hbase
> transactions)
>
> [image: cid:image001.jpg@01D1011A.5E93FF70]
>
>
>
> - After Hbase restarting at June 18th, The spikes had been reduced
> dramatically. But its ratio had been increasing continuously.
>
> [image: cid:image002.png@01D1011A.5E93FF70]
>
>
>
> - We had adopted off-heap bucket cache at August 19th. But the increasing
> does not disappear.
>
> [image: cid:image002.png@01D10103.82E488C0]
>
> [image: cid:image002.png@01D10103.82E488C0]
>
>
>
> - We set HBase to log slow query which have above 10ms execution time. All
> RegionServer logs show increasing its counts.
>
>
>
> [image: cid:image005.png@01D1011A.5E93FF70]
>
>
>
>
>
> What we concern is not the spike itself but its incremental trend.
>
> There is no increment of transactions and dataset.
>
> I ask your thoughts about this issue.
>
>
>
>
>
>
>
>
>
> -------------------------------------------------------------
>
> *UI JIN, LIM*
>
> *DNA 3.0** - **LG CNS*
>
> Data Architecture Advisory
>
> Big Data Technical Support Team
>
> Mobile : +82) 010-8307-1210
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message