Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Content-Type: multipart/alternative;
	boundary=Apple-Mail-8CB1B1B2-FEBA-4153-9D37-0367C30C9129
Mime-Version: 1.0 (1.0)
Subject: Re: Use experience and performance data of offheap from Alibaba online cluster
From: Ted Yu <yuzhihong@gmail.com>
In-Reply-To: <CAM7-19+n7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA@mail.gmail.com>
Date: Fri, 18 Nov 2016 03:29:22 -0800
Cc: user@hbase.apache.org
Content-Transfer-Encoding: 7bit
Message-Id: <943F698A-7CD9-4D65-B083-D2F7ED570A0E@gmail.com>
References: <CAM7-19+n7cEiY4H9iLQ3N9V0NXppOPduZwk-hhgNLEaJfiV3kA@mail.gmail.com>
To: dev@hbase.apache.org
archived-at: Fri, 18 Nov 2016 11:29:36 -0000

--Apple-Mail-8CB1B1B2-FEBA-4153-9D37-0367C30C9129
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable

Yu:
With positive results, more hbase users would be asking for the backport of o=
ffheap read path patches.=20

Do you think you or your coworker has the bandwidth to publish backport for b=
ranch-1 ?

Thanks=20

> On Nov 18, 2016, at 12:11 AM, Yu Li <carp84@gmail.com> wrote:
>=20
> Dear all,
>=20
> We have backported read path offheap (HBASE-11425) to our customized hbase=
-1.1.2 (thanks @Anoop for the help/support) and run it online for more than a=
 month, and would like to share our experience, for what it's worth (smile).=

>=20
> Generally speaking, we gained a better and more stable throughput/performa=
nce with offheap, and below are some details:
> 1. QPS become more stable with offheap
>=20
> Performance w/o offheap:
>=20
>=20
>=20
> Performance w/ offheap:
>=20
>=20
>=20
> These data come from our online A/B test cluster (with 450 physical machin=
es, and each with 256G memory + 64 core) with real world workloads, it shows=
 using offheap we could gain a more stable throughput as well as better perf=
ormance
>=20
> Not showing fully online data here because for online we published the ver=
sion with both offheap and NettyRpcServer together, so no standalone compari=
son data for offheap
>=20
> 2. Full GC frequency and cost
>=20
> Average Full GC STW time reduce from 11s to 7s with offheap.
>=20
> 3. Young GC frequency and cost
>=20
> No performance degradation observed with offheap.
>=20
> 4. Peak throughput of one single RS
>=20
> On Singles Day (11/11), peak throughput of one single RS reached 100K, amo=
ng which 90K from Get. Plus internet in/out data we could know the average r=
esult size of get request is ~1KB
>=20
>=20
>=20
> Offheap are used on all online machines (more than 1600 nodes) instead of L=
ruCache, so the above QPS is gained from offheap bucketcache, along with Net=
tyRpcServer(HBASE-15756).
>=20
> Just let us know if any comments. Thanks.
>=20
> Best Regards,
> Yu
>=20
>=20
>=20
>=20
>=20
>=20
>=20

--Apple-Mail-8CB1B1B2-FEBA-4153-9D37-0367C30C9129--