hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elliott Clark <ecl...@stumbleupon.com>
Subject Re: ANN: The third hbase 0.94.0 release candidate is available for download
Date Fri, 04 May 2012 22:04:37 GMT
So I got 94.0rc3 up on a cluster and tried to break it, Killing masters and
killing rs.  Everything seems good. hbck reports everything is good.  And
all my reads succeed.

I'll post cluster benchmark numbers once they are done running.  Should
only be a couple more hours of pe runs.

Looks great to me.
On Thu, May 3, 2012 at 10:36 AM, Elliott Clark <eclark@stumbleupon.com>wrote:

> I agree it was just a micro benchmark with no guarantee that it relates to
> real world. With it just being standalone I didn't think anyone should take
> the numbers as 100% representative.  Really I was just trying to shake out
> any weird behaviors and the fact that we got a big speed up was
> interesting.
>
>
> On Thu, May 3, 2012 at 12:15 AM, Mikael Sitruk <mikael.sitruk@gmail.com>wrote:
>
>> Hi guys
>> Looking at the posted slide/pictures for the benchmark the
>> following intriguing me:
>> 1. The recordcount is only 100,000
>> 2. workoloada is: read 50%, update 50% and zipfian distribution even with
>> 5M operations count, the same keys are updated again and again.
>> 3. heap size 10G
>>
>> Therefore it might be that the dataset is too small (even with 3 versions
>> configured we have = 3(version)*100,000(keys)*1KB (size of record) = 300
>> MB
>> of "live" dataset ?
>> And approximately the number of store files will be 5x10^6 (op
>> count)*1KB(record size)/256MB(max store file size (Default))=>20 store
>> file, even taking factor of 10 for metadata (record key, in store files)
>> we
>> will get 200 files.
>> if a major compaction is running it will shrink all the storefile to a
>> single small one.
>> What I try to say is - if the maths are correct - (please note that i did
>> not take into account compression which just make things better), can we
>> relate on such scenario for performance benchmark with such small dataset
>> and such distribution?
>>
>> Regards
>> Mikael.S
>>
>> On Thu, May 3, 2012 at 1:10 AM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>> > I am surprised to see 0.92.1 exhibit such unfavorable performance
>> profile.
>> > Let's see whether cluster testing gives us similar results.
>> >
>> > On Wed, May 2, 2012 at 3:07 PM, Elliott Clark <eclark@stumbleupon.com
>> > >wrote:
>> >
>> > > Sure, sorry about that.
>> > >
>> > > http://imgur.com/waxlS
>> > >
>> > >
>> >
>> http://www.scribd.com/eclark847297/d/92151092-Hbase-0-94-0-RC3-Local-YCSB-Perf
>> > >
>> > > On Wed, May 2, 2012 at 3:01 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> > >
>> > > > Elliot:
>> > > > Thanks for the report.
>> > > > Can you publish results somewhere else ?
>> > > > Attachments were stripped off.
>> > > >
>> > > > On Wed, May 2, 2012 at 2:59 PM, Elliott Clark <
>> eclark@stumbleupon.com
>> > > > >wrote:
>> > > >
>> > > > > I ran some tests of local filesystem YCSB. I used the 0.90 client
>> for
>> > > > > 0.90.6.  For the rest of the tests I used 0.92 clients. The
>> results
>> > are
>> > > > > attached.
>> > > > >
>> > > > > 0.90 -> 0.94.0RC3 13% faster
>> > > > > 0.92 -> 0.94.0RC3 50% faster
>> > > > >
>> > > > >  This seems to be a pretty large performance improvement.  I'll
>> run
>> > > some
>> > > > > tests on a cluster later today.
>> > > > >
>> > > > > On Tue, May 1, 2012 at 10:20 PM, lars hofhansl <
>> lhofhansl@yahoo.com
>> > > > >wrote:
>> > > > >
>> > > > >> Thanks Todd.
>> > > > >>
>> > > > >> I agree with doing source code releases going forward.
>> > > > >>
>> > > > >> For that, would it be sufficient to just vote against an
SVN tag?
>> > > > >> Tarballs can then be pulled straight from that tag.
>> > > > >>
>> > > > >> -- Lars
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >> ----- Original Message -----
>> > > > >> From: Todd Lipcon <todd@cloudera.com>
>> > > > >> To: dev@hbase.apache.org; lars hofhansl <lhofhansl@yahoo.com>
>> > > > >> Cc:
>> > > > >> Sent: Tuesday, May 1, 2012 9:35 PM
>> > > > >> Subject: Re: ANN: The third hbase 0.94.0 release candidate
is
>> > > available
>> > > > >> for download
>> > > > >>
>> > > > >> +1 from me, I took it for a spin on the local filesystem
with
>> some
>> > > YCSB
>> > > > >> load.
>> > > > >>
>> > > > >> Here is my signature on the non-secure tarball.
>> > > > >> -----BEGIN PGP SIGNATURE-----
>> > > > >> Version: GnuPG v1.4.10 (GNU/Linux)
>> > > > >>
>> > > > >> iEYEABECAAYFAk+guTIACgkQXkPKua7Hfq9YSQCeMnCQ4XFqLjw+PF8IXNPDug+t
>> > > > >> h90AoJ+q4YSg4JbfiCmaXenadWSRU1of
>> > > > >> =CdfZ
>> > > > >> -----END PGP SIGNATURE-----
>> > > > >>
>> > > > >> I didn't check out the secure tarball.
>> > > > >>
>> > > > >> I think for future releases we should do the voting against
a
>> source
>> > > tar
>> > > > >> (ie an svn export) since we now produce multiple binaries,
and
>> it's
>> > > > easier
>> > > > >> to verify that a source tar matches SVN, etc.
>> > > > >>
>> > > > >> -Todd
>> > > > >>
>> > > > >>
>> > > > >> On Tue, May 1, 2012 at 4:26 PM, lars hofhansl <
>> lhofhansl@yahoo.com>
>> > > > >> wrote:
>> > > > >>
>> > > > >> > The third 0.94.0 RC is available for download here:
>> > > > >> > http://people.apache.org/~larsh/hbase-0.94.0-rc3/
>> > > > >> > (My gpg key is available from pgp.mit.edu. Key id: 7CA45750)
>> > > > >> >
>> > > > >> > HBase 0.94 is a performance release, and there are some
>> > interesting
>> > > > new
>> > > > >> > features as well.
>> > > > >> >
>> > > > >> > It is wire compatible with 0.92.x. 0.92 clients should
work
>> with
>> > > 0.94
>> > > > >> > servers and vice versa.
>> > > > >> >
>> > > > >> > You can do a rolling restart to get your 0.92.x HBase
up on
>> this
>> > > > >> 0.94.0RC.
>> > > > >> >
>> > > > >> > The full list of changes is available here:
>> > > > >> >
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12316419
>> > > > >> >
>> > > > >> > Please take this RC for a spin, check out the doc, etc,
and
>> vote
>> > > +1/-1
>> > > > >> by
>> > > > >> > May 8th on whether we should release this as 0.94.0.
>> > > > >> >
>> > > > >> > Thanks.
>> > > > >> >
>> > > > >> > -- Lars
>> > > > >> >
>> > > > >>
>> > > > >>
>> > > > >>
>> > > > >> --
>> > > > >> Todd Lipcon
>> > > > >> Software Engineer, Cloudera
>> > > > >>
>> > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message