hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: 0.92 and Read/writes not scaling
Date Thu, 05 Apr 2012 02:43:15 GMT
Thanks for sharing your results.

Do you mind putting the summary on HBASE-5699: Run with > 1 WAL in
HRegionServer ?

On Wed, Apr 4, 2012 at 6:45 PM, Juhani Connolly <
juhani_connolly@cyberagent.co.jp> wrote:

> another quick update on stuff:
> since moving back to hdfs 0.20.2 (with hbase still at 0.92), we found that
> while we made significant gains in throughput, that most of our
> regionservers IPC threads were stuck somewhere in HWal.append(out of 50, 42
> were in append, of which 20 were in sync), limiting throughput despite
> significant free hardware resources.
> Because the WAL writes of a single  RS all go sequentially to one HDFS
> file, we assumed that we could improve throughput by separating writes to
> more WAL files and more HDs. To do this we ran multiple region servers on
> each node.
> The scaling  wasn't linear(we were in no way increasing hardware, just the
> number of regionservers), but we are now getting significantly more
> throughput.
> I would personally not say that this is a great approach to have to take,
> it would generally be better to build more smaller servers which will thus
> not limit themselves by trying to put a lot of data per server through a
> single WAL file.
> Of course there may be another solution to this that I'm not aware of? If
> so I'd love to hear it.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message