Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of joey@cloudera.com designates
 209.85.214.169 as permitted sender)
Subject: Re: performance: HLog flush to disk each thread,
 can we decrease IO calls?
References: <BANLkTinhkbknJ+PjEzm3Vf96gXtL+wsiXA@mail.gmail.com>
From: Joey Echeverria <joey@cloudera.com>
Content-Type: multipart/alternative;
	boundary=Apple-Mail-1-333207594
In-Reply-To: <BANLkTinhkbknJ+PjEzm3Vf96gXtL+wsiXA@mail.gmail.com>
Message-Id: <B60BA63A-5159-48E3-AF5A-5F3777EB1243@cloudera.com>
Date: Wed, 29 Jun 2011 06:08:06 -0700
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (iPad Mail 8G4)

--Apple-Mail-1-333207594
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

Hey Mingjian,

This sounds like a good idea  Your patch didn't make it through. Would you m=
ind either filing a JIRA and uploading your patch there or at least posting i=
t to something like pastebin so we can take a look.

-Joey


On Jun 29, 2011, at 3:27, Mingjian Deng <koven2049@gmail.com> wrote:

> Hi:
>     We found that the hlog sync to disk each time. When one thread exec "d=
oWrite(info, logKey, edit);", the others wait for "updateLock" in HLog.java.=

>     Why not the others add their edits into a list and wait. When sync's t=
ime, the whole list sync to disk once. I think it will decrease the IO calls=
.=20
>=20
>     So Maybe we will make two lists for edits. Each thread write to the "w=
aledits" and wait for "updateLock". Each thread can copy the "waledits" to "=
flushedits" and flush the "flushedits" to=20
> disk once it gets "updateLock".
>=20
>     In my test, it can increase the write speed of 40%.
>=20
>     Just see the HLog.patch.
>    =20

--Apple-Mail-1-333207594--