Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of binlijin@gmail.com designates
 209.85.212.179 as permitted sender)
MIME-Version: 1.0
Date: Sun, 29 Jul 2012 23:09:01 +0800
Message-ID: 
 <CAHNAVbud79Gsv2pd07gDe9xeg4r0=wbt9uUsGFjE=7G5YQ+Kgw@mail.gmail.com>
Subject: 
 =?GB2312?Q?DFSClient_Packet=A1=AEis_not_fully_used=A3=ACintroduced_Pa?=
	=?GB2312?Q?cket_Pool_and_reuse_Packet=2E?=
From: =?GB2312?B?sfbA8r3w?= <binlijin@gmail.com>
To: dev@hbase.apache.org
Content-Type: multipart/alternative; boundary=f46d044481d74db8e804c5f95340

--f46d044481d74db8e804c5f95340
Content-Type: text/plain; charset=GB2312
Content-Transfer-Encoding: quoted-printable

hi,  all devs
    Current when FSDataOutputStream's write and sync method are called a
new Packet may be created and put into dataQueue, the daemon DataStreamer
will get packet from the dataQueue and write to datanode.
    I find every Packet will new a byte array with size is 65557(more than
64K). but the pktLen is just a few hundreds or a few ten hunderds . So new
a byte array with big size but just use a very little of it. So the
utilization
rate is very low.
   If 1024 Packet are created one second=A3=AC64M+ garbage is produced one
second.
   I rewrite the logic of new Packet with a PacketPool=A3=ACI find the
hregionserver's gc frequency and gc total time reduce a lot=A3=ACbut a sing=
le gc
time increase a lot=A3=ACso i reduce the jvm New gen size=A3=ACbut the same=
 result
get=A3=ACThe average RT increase and TPS reduced. I cann't see why? And wis=
h
some one can told me why? may be the reason is gc.
  Although average RT increase=A3=ACbut more request has little rt and also=
 more
request has a big rt=A3=ACso the  average RT increase.


--=20
*Best Regards,*
 lijin bin

--f46d044481d74db8e804c5f95340--