hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: ANN: The third hbase 0.94.0 release candidate is available for download
Date Tue, 08 May 2012 20:21:33 GMT
Hmm... So our "performance release" is slightly slower than 0.92.
With all the optimizations that went into 0.94 I find that a bit hard to believe.

Can you tell us more about the testing? How many machines, setup, was that test IO or CPU
bound, etc?
Anything else of note?

Thanks for doing this!

-- Lars

________________________________
From: Elliott Clark <eclark@stumbleupon.com>
To: dev@hbase.apache.org 
Sent: Monday, May 7, 2012 11:07 AM
Subject: Re: ANN: The third hbase 0.94.0 release candidate is available for download

Sorry everything is in elapsed time as reported by Elapsed time in
milliseconds.  So higher is worse.

The standard deviation on 0.92.1 writes is 4,591,384 so Write 5 is a little
outside of 1 std dev.  Not really sure what happened on that test, but it
does appear that PE is very noisy.

On Mon, May 7, 2012 at 10:47 AM, Todd Lipcon <todd@cloudera.com> wrote:

> Is higher better or worse? :) Any idea what happened on the "Write 5" test?
>
> On Mon, May 7, 2012 at 10:42 AM, Elliott Clark <eclark@stumbleupon.com>
> wrote:
> > http://www.scribd.com/eclark847297/d/92715238-0-94-0-RC3-Cluster-Perf
> >
> > On Fri, May 4, 2012 at 7:42 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> >> 0.94 also has LoadTestTool (from FB)
> >>
> >> I have used it to do some cluster load testing.
> >>
> >> Just FYI
> >>
> >> On Fri, May 4, 2012 at 3:14 PM, Elliott Clark <eclark@stumbleupon.com
> >> >wrote:
> >>
> >> > With the cluster size that I'm testing YCSB was stressing the client
> >> > machine more than the cluster.  I was saturating the network of the
> test
> >> > machine.  So I switched over to pe; while it doesn't have a realistic
> >> work
> >> > load it is better than nothing.
> >> >
> >> > On Fri, May 4, 2012 at 3:07 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >> >
> >> > > Thanks for the update, Elliot.
> >> > >
> >> > > If I read your post correctly, you're using PE. ycsb is better
> >> measuring
> >> > > performance, from my experience.
> >> > >
> >> > > Cheers
> >> > >
> >> > > On Fri, May 4, 2012 at 3:04 PM, Elliott Clark <
> eclark@stumbleupon.com
> >> > > >wrote:
> >> > >
> >> > > > So I got 94.0rc3 up on a cluster and tried to break it, Killing
> >> masters
> >> > > and
> >> > > > killing rs.  Everything seems good. hbck reports everything
is
> good.
> >> >  And
> >> > > > all my reads succeed.
> >> > > >
> >> > > > I'll post cluster benchmark numbers once they are done running.
> >>  Should
> >> > > > only be a couple more hours of pe runs.
> >> > > >
> >> > > > Looks great to me.
> >> > > > On Thu, May 3, 2012 at 10:36 AM, Elliott Clark <
> >> eclark@stumbleupon.com
> >> > > > >wrote:
> >> > > >
> >> > > > > I agree it was just a micro benchmark with no guarantee
that it
> >> > relates
> >> > > > to
> >> > > > > real world. With it just being standalone I didn't think
anyone
> >> > should
> >> > > > take
> >> > > > > the numbers as 100% representative.  Really I was just
trying to
> >> > shake
> >> > > > out
> >> > > > > any weird behaviors and the fact that we got a big speed
up was
> >> > > > > interesting.
> >> > > > >
> >> > > > >
> >> > > > > On Thu, May 3, 2012 at 12:15 AM, Mikael Sitruk <
> >> > > mikael.sitruk@gmail.com
> >> > > > >wrote:
> >> > > > >
> >> > > > >> Hi guys
> >> > > > >> Looking at the posted slide/pictures for the benchmark
the
> >> > > > >> following intriguing me:
> >> > > > >> 1. The recordcount is only 100,000
> >> > > > >> 2. workoloada is: read 50%, update 50% and zipfian distribution
> >> even
> >> > > > with
> >> > > > >> 5M operations count, the same keys are updated again
and again.
> >> > > > >> 3. heap size 10G
> >> > > > >>
> >> > > > >> Therefore it might be that the dataset is too small
(even with
> 3
> >> > > > versions
> >> > > > >> configured we have = 3(version)*100,000(keys)*1KB (size
of
> >> record) =
> >> > > 300
> >> > > > >> MB
> >> > > > >> of "live" dataset ?
> >> > > > >> And approximately the number of store files will be
5x10^6 (op
> >> > > > >> count)*1KB(record size)/256MB(max store file size
> (Default))=>20
> >> > store
> >> > > > >> file, even taking factor of 10 for metadata (record
key, in
> store
> >> > > files)
> >> > > > >> we
> >> > > > >> will get 200 files.
> >> > > > >> if a major compaction is running it will shrink all
the
> storefile
> >> > to a
> >> > > > >> single small one.
> >> > > > >> What I try to say is - if the maths are correct - (please
note
> >> that
> >> > i
> >> > > > did
> >> > > > >> not take into account compression which just make things
> better),
> >> > can
> >> > > we
> >> > > > >> relate on such scenario for performance benchmark with
such
> small
> >> > > > dataset
> >> > > > >> and such distribution?
> >> > > > >>
> >> > > > >> Regards
> >> > > > >> Mikael.S
> >> > > > >>
> >> > > > >> On Thu, May 3, 2012 at 1:10 AM, Ted Yu <yuzhihong@gmail.com>
> >> wrote:
> >> > > > >>
> >> > > > >> > I am surprised to see 0.92.1 exhibit such unfavorable
> >> performance
> >> > > > >> profile.
> >> > > > >> > Let's see whether cluster testing gives us similar
results.
> >> > > > >> >
> >> > > > >> > On Wed, May 2, 2012 at 3:07 PM, Elliott Clark <
> >> > > eclark@stumbleupon.com
> >> > > > >> > >wrote:
> >> > > > >> >
> >> > > > >> > > Sure, sorry about that.
> >> > > > >> > >
> >> > > > >> > > http://imgur.com/waxlS
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> http://www.scribd.com/eclark847297/d/92151092-Hbase-0-94-0-RC3-Local-YCSB-Perf
> >> > > > >> > >
> >> > > > >> > > On Wed, May 2, 2012 at 3:01 PM, Ted Yu <
> yuzhihong@gmail.com>
> >> > > wrote:
> >> > > > >> > >
> >> > > > >> > > > Elliot:
> >> > > > >> > > > Thanks for the report.
> >> > > > >> > > > Can you publish results somewhere else
?
> >> > > > >> > > > Attachments were stripped off.
> >> > > > >> > > >
> >> > > > >> > > > On Wed, May 2, 2012 at 2:59 PM, Elliott
Clark <
> >> > > > >> eclark@stumbleupon.com
> >> > > > >> > > > >wrote:
> >> > > > >> > > >
> >> > > > >> > > > > I ran some tests of local filesystem
YCSB. I used the
> 0.90
> >> > > > client
> >> > > > >> for
> >> > > > >> > > > > 0.90.6.  For the rest of the tests
I used 0.92 clients.
> >> The
> >> > > > >> results
> >> > > > >> > are
> >> > > > >> > > > > attached.
> >> > > > >> > > > >
> >> > > > >> > > > > 0.90 -> 0.94.0RC3 13% faster
> >> > > > >> > > > > 0.92 -> 0.94.0RC3 50% faster
> >> > > > >> > > > >
> >> > > > >> > > > >  This seems to be a pretty large
performance
> improvement.
> >> > >  I'll
> >> > > > >> run
> >> > > > >> > > some
> >> > > > >> > > > > tests on a cluster later today.
> >> > > > >> > > > >
> >> > > > >> > > > > On Tue, May 1, 2012 at 10:20 PM,
lars hofhansl <
> >> > > > >> lhofhansl@yahoo.com
> >> > > > >> > > > >wrote:
> >> > > > >> > > > >
> >> > > > >> > > > >> Thanks Todd.
> >> > > > >> > > > >>
> >> > > > >> > > > >> I agree with doing source code
releases going forward.
> >> > > > >> > > > >>
> >> > > > >> > > > >> For that, would it be sufficient
to just vote against
> an
> >> > SVN
> >> > > > tag?
> >> > > > >> > > > >> Tarballs can then be pulled
straight from that tag.
> >> > > > >> > > > >>
> >> > > > >> > > > >> -- Lars
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > >> > > > >> ----- Original Message -----
> >> > > > >> > > > >> From: Todd Lipcon <todd@cloudera.com>
> >> > > > >> > > > >> To: dev@hbase.apache.org; lars
hofhansl <
> >> > lhofhansl@yahoo.com
> >> > > >
> >> > > > >> > > > >> Cc:
> >> > > > >> > > > >> Sent: Tuesday, May 1, 2012 9:35
PM
> >> > > > >> > > > >> Subject: Re: ANN: The third
hbase 0.94.0 release
> >> candidate
> >> > is
> >> > > > >> > > available
> >> > > > >> > > > >> for download
> >> > > > >> > > > >>
> >> > > > >> > > > >> +1 from me, I took it for a
spin on the local
> filesystem
> >> > with
> >> > > > >> some
> >> > > > >> > > YCSB
> >> > > > >> > > > >> load.
> >> > > > >> > > > >>
> >> > > > >> > > > >> Here is my signature on the
non-secure tarball.
> >> > > > >> > > > >> -----BEGIN PGP SIGNATURE-----
> >> > > > >> > > > >> Version: GnuPG v1.4.10 (GNU/Linux)
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > iEYEABECAAYFAk+guTIACgkQXkPKua7Hfq9YSQCeMnCQ4XFqLjw+PF8IXNPDug+t
> >> > > > >> > > > >> h90AoJ+q4YSg4JbfiCmaXenadWSRU1of
> >> > > > >> > > > >> =CdfZ
> >> > > > >> > > > >> -----END PGP SIGNATURE-----
> >> > > > >> > > > >>
> >> > > > >> > > > >> I didn't check out the secure
tarball.
> >> > > > >> > > > >>
> >> > > > >> > > > >> I think for future releases
we should do the voting
> >> > against a
> >> > > > >> source
> >> > > > >> > > tar
> >> > > > >> > > > >> (ie an svn export) since we
now produce multiple
> >> binaries,
> >> > > and
> >> > > > >> it's
> >> > > > >> > > > easier
> >> > > > >> > > > >> to verify that a source tar
matches SVN, etc.
> >> > > > >> > > > >>
> >> > > > >> > > > >> -Todd
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > >> > > > >> On Tue, May 1, 2012 at 4:26
PM, lars hofhansl <
> >> > > > >> lhofhansl@yahoo.com>
> >> > > > >> > > > >> wrote:
> >> > > > >> > > > >>
> >> > > > >> > > > >> > The third 0.94.0 RC is
available for download here:
> >> > > > >> > > > >> > http://people.apache.org/~larsh/hbase-0.94.0-rc3/
> >> > > > >> > > > >> > (My gpg key is available
from pgp.mit.edu. Key id:
> >> > > 7CA45750)
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > HBase 0.94 is a performance
release, and there are
> some
> >> > > > >> > interesting
> >> > > > >> > > > new
> >> > > > >> > > > >> > features as well.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > It is wire compatible with
0.92.x. 0.92 clients
> should
> >> > work
> >> > > > >> with
> >> > > > >> > > 0.94
> >> > > > >> > > > >> > servers and vice versa.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > You can do a rolling restart
to get your 0.92.x
> HBase
> >> up
> >> > on
> >> > > > >> this
> >> > > > >> > > > >> 0.94.0RC.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > The full list of changes
is available here:
> >> > > > >> > > > >> >
> >> > > > >> > > > >> >
> >> > > > >> > > > >>
> >> > > > >> > > >
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12316419
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > Please take this RC for
a spin, check out the doc,
> etc,
> >> > and
> >> > > > >> vote
> >> > > > >> > > +1/-1
> >> > > > >> > > > >> by
> >> > > > >> > > > >> > May 8th on whether we should
release this as 0.94.0.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > Thanks.
> >> > > > >> > > > >> >
> >> > > > >> > > > >> > -- Lars
> >> > > > >> > > > >> >
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > >> > > > >> --
> >> > > > >> > > > >> Todd Lipcon
> >> > > > >> > > > >> Software Engineer, Cloudera
> >> > > > >> > > > >>
> >> > > > >> > > > >>
> >> > > > >> > > > >
> >> > > > >> > > >
> >> > > > >> > >
> >> > > > >> >
> >> > > > >>
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Mime
View raw message