hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joey Echeverria <j...@cloudera.com>
Subject Re: performance: HLog flush to disk each thread, can we decrease IO calls?
Date Wed, 29 Jun 2011 13:08:06 GMT
Hey Mingjian,

This sounds like a good idea  Your patch didn't make it through. Would you mind either filing
a JIRA and uploading your patch there or at least posting it to something like pastebin so
we can take a look.


On Jun 29, 2011, at 3:27, Mingjian Deng <koven2049@gmail.com> wrote:

> Hi:
>     We found that the hlog sync to disk each time. When one thread exec "doWrite(info,
logKey, edit);", the others wait for "updateLock" in HLog.java.
>     Why not the others add their edits into a list and wait. When sync's time, the whole
list sync to disk once. I think it will decrease the IO calls. 
>     So Maybe we will make two lists for edits. Each thread write to the "waledits" and
wait for "updateLock". Each thread can copy the "waledits" to "flushedits" and flush the "flushedits"
> disk once it gets "updateLock".
>     In my test, it can increase the write speed of 40%.
>     Just see the HLog.patch.

  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message