Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of eclark@stumbleupon.com
 designates 74.125.82.48 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALte62y+m4UmrCnXGkckLZzNaR1ZhQ-V2Z1vPOAq-Th+qSF8FA@mail.gmail.com>
References: <1335914771.79923.YahooMailNeo@web121706.mail.ne1.yahoo.com>
	<CADY20s7rmTinW5m3q2dvMhPPj9e_7pu7T8wVwpWBmHGVQjASVA@mail.gmail.com>
	<1335936048.37273.YahooMailNeo@web121703.mail.ne1.yahoo.com>
	<CAKYwJ9ytcQKOtDin776TpoJ8ceuad16DscWKQzpM45=BNRjO4Q@mail.gmail.com>
	<CALte62ybPHr1UHXY8NPXP6UyJejWvjV=Y5rWVKx3BE1JxOG-RQ@mail.gmail.com>
	<CAKYwJ9w-K6uHZ2gjACiT6ci_hHsGLuJxWOeLZ+QKU4fOH+odYQ@mail.gmail.com>
	<CALte62z_Son4ZcW7Veh2ZQ7qKqsZszaFYRdVixJf9p6HJStm5g@mail.gmail.com>
	<CAPpvZ9Bxy44m-SDvMtGdSVzr1WdjS5yf=-2K61AXGSOfqMZeMQ@mail.gmail.com>
	<CAKYwJ9yFm8nCpc7+Ann_yLCVim_zzwgZHgQj_iFFtQNJzRAZJg@mail.gmail.com>
	<CAKYwJ9yPsJtiD3BvmOqMq+GGS4Z2J2fONGkjk1AGEO576ZSoZA@mail.gmail.com>
	<CALte62y+m4UmrCnXGkckLZzNaR1ZhQ-V2Z1vPOAq-Th+qSF8FA@mail.gmail.com>
Date: Fri, 4 May 2012 15:14:03 -0700
Message-ID: 
 <CAKYwJ9zXFNn60xg71ueZ5gnbNRSfyigWnFzjHnQJMr9PW=6Cmw@mail.gmail.com>
Subject: Re: ANN: The third hbase 0.94.0 release candidate is available for
 download
From: Elliott Clark <eclark@stumbleupon.com>
To: dev@hbase.apache.org
Content-Type: multipart/alternative; boundary=f46d043c81ecf58d2e04bf3d3c1d

--f46d043c81ecf58d2e04bf3d3c1d
Content-Type: text/plain; charset=ISO-8859-1

With the cluster size that I'm testing YCSB was stressing the client
machine more than the cluster.  I was saturating the network of the test
machine.  So I switched over to pe; while it doesn't have a realistic work
load it is better than nothing.

On Fri, May 4, 2012 at 3:07 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Thanks for the update, Elliot.
>
> If I read your post correctly, you're using PE. ycsb is better measuring
> performance, from my experience.
>
> Cheers
>
> On Fri, May 4, 2012 at 3:04 PM, Elliott Clark <eclark@stumbleupon.com
> >wrote:
>
> > So I got 94.0rc3 up on a cluster and tried to break it, Killing masters
> and
> > killing rs.  Everything seems good. hbck reports everything is good.  And
> > all my reads succeed.
> >
> > I'll post cluster benchmark numbers once they are done running.  Should
> > only be a couple more hours of pe runs.
> >
> > Looks great to me.
> > On Thu, May 3, 2012 at 10:36 AM, Elliott Clark <eclark@stumbleupon.com
> > >wrote:
> >
> > > I agree it was just a micro benchmark with no guarantee that it relates
> > to
> > > real world. With it just being standalone I didn't think anyone should
> > take
> > > the numbers as 100% representative.  Really I was just trying to shake
> > out
> > > any weird behaviors and the fact that we got a big speed up was
> > > interesting.
> > >
> > >
> > > On Thu, May 3, 2012 at 12:15 AM, Mikael Sitruk <
> mikael.sitruk@gmail.com
> > >wrote:
> > >
> > >> Hi guys
> > >> Looking at the posted slide/pictures for the benchmark the
> > >> following intriguing me:
> > >> 1. The recordcount is only 100,000
> > >> 2. workoloada is: read 50%, update 50% and zipfian distribution even
> > with
> > >> 5M operations count, the same keys are updated again and again.
> > >> 3. heap size 10G
> > >>
> > >> Therefore it might be that the dataset is too small (even with 3
> > versions
> > >> configured we have = 3(version)*100,000(keys)*1KB (size of record) =
> 300
> > >> MB
> > >> of "live" dataset ?
> > >> And approximately the number of store files will be 5x10^6 (op
> > >> count)*1KB(record size)/256MB(max store file size (Default))=>20 store
> > >> file, even taking factor of 10 for metadata (record key, in store
> files)
> > >> we
> > >> will get 200 files.
> > >> if a major compaction is running it will shrink all the storefile to a
> > >> single small one.
> > >> What I try to say is - if the maths are correct - (please note that i
> > did
> > >> not take into account compression which just make things better), can
> we
> > >> relate on such scenario for performance benchmark with such small
> > dataset
> > >> and such distribution?
> > >>
> > >> Regards
> > >> Mikael.S
> > >>
> > >> On Thu, May 3, 2012 at 1:10 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> > >>
> > >> > I am surprised to see 0.92.1 exhibit such unfavorable performance
> > >> profile.
> > >> > Let's see whether cluster testing gives us similar results.
> > >> >
> > >> > On Wed, May 2, 2012 at 3:07 PM, Elliott Clark <
> eclark@stumbleupon.com
> > >> > >wrote:
> > >> >
> > >> > > Sure, sorry about that.
> > >> > >
> > >> > > http://imgur.com/waxlS
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> http://www.scribd.com/eclark847297/d/92151092-Hbase-0-94-0-RC3-Local-YCSB-Perf
> > >> > >
> > >> > > On Wed, May 2, 2012 at 3:01 PM, Ted Yu <yuzhihong@gmail.com>
> wrote:
> > >> > >
> > >> > > > Elliot:
> > >> > > > Thanks for the report.
> > >> > > > Can you publish results somewhere else ?
> > >> > > > Attachments were stripped off.
> > >> > > >
> > >> > > > On Wed, May 2, 2012 at 2:59 PM, Elliott Clark <
> > >> eclark@stumbleupon.com
> > >> > > > >wrote:
> > >> > > >
> > >> > > > > I ran some tests of local filesystem YCSB. I used the 0.90
> > client
> > >> for
> > >> > > > > 0.90.6.  For the rest of the tests I used 0.92 clients. The
> > >> results
> > >> > are
> > >> > > > > attached.
> > >> > > > >
> > >> > > > > 0.90 -> 0.94.0RC3 13% faster
> > >> > > > > 0.92 -> 0.94.0RC3 50% faster
> > >> > > > >
> > >> > > > >  This seems to be a pretty large performance improvement.
>  I'll
> > >> run
> > >> > > some
> > >> > > > > tests on a cluster later today.
> > >> > > > >
> > >> > > > > On Tue, May 1, 2012 at 10:20 PM, lars hofhansl <
> > >> lhofhansl@yahoo.com
> > >> > > > >wrote:
> > >> > > > >
> > >> > > > >> Thanks Todd.
> > >> > > > >>
> > >> > > > >> I agree with doing source code releases going forward.
> > >> > > > >>
> > >> > > > >> For that, would it be sufficient to just vote against an SVN
> > tag?
> > >> > > > >> Tarballs can then be pulled straight from that tag.
> > >> > > > >>
> > >> > > > >> -- Lars
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > > > >> ----- Original Message -----
> > >> > > > >> From: Todd Lipcon <todd@cloudera.com>
> > >> > > > >> To: dev@hbase.apache.org; lars hofhansl <lhofhansl@yahoo.com
> >
> > >> > > > >> Cc:
> > >> > > > >> Sent: Tuesday, May 1, 2012 9:35 PM
> > >> > > > >> Subject: Re: ANN: The third hbase 0.94.0 release candidate is
> > >> > > available
> > >> > > > >> for download
> > >> > > > >>
> > >> > > > >> +1 from me, I took it for a spin on the local filesystem with
> > >> some
> > >> > > YCSB
> > >> > > > >> load.
> > >> > > > >>
> > >> > > > >> Here is my signature on the non-secure tarball.
> > >> > > > >> -----BEGIN PGP SIGNATURE-----
> > >> > > > >> Version: GnuPG v1.4.10 (GNU/Linux)
> > >> > > > >>
> > >> > > > >>
> > iEYEABECAAYFAk+guTIACgkQXkPKua7Hfq9YSQCeMnCQ4XFqLjw+PF8IXNPDug+t
> > >> > > > >> h90AoJ+q4YSg4JbfiCmaXenadWSRU1of
> > >> > > > >> =CdfZ
> > >> > > > >> -----END PGP SIGNATURE-----
> > >> > > > >>
> > >> > > > >> I didn't check out the secure tarball.
> > >> > > > >>
> > >> > > > >> I think for future releases we should do the voting against a
> > >> source
> > >> > > tar
> > >> > > > >> (ie an svn export) since we now produce multiple binaries,
> and
> > >> it's
> > >> > > > easier
> > >> > > > >> to verify that a source tar matches SVN, etc.
> > >> > > > >>
> > >> > > > >> -Todd
> > >> > > > >>
> > >> > > > >>
> > >> > > > >> On Tue, May 1, 2012 at 4:26 PM, lars hofhansl <
> > >> lhofhansl@yahoo.com>
> > >> > > > >> wrote:
> > >> > > > >>
> > >> > > > >> > The third 0.94.0 RC is available for download here:
> > >> > > > >> > http://people.apache.org/~larsh/hbase-0.94.0-rc3/
> > >> > > > >> > (My gpg key is available from pgp.mit.edu. Key id:
> 7CA45750)
> > >> > > > >> >
> > >> > > > >> > HBase 0.94 is a performance release, and there are some
> > >> > interesting
> > >> > > > new
> > >> > > > >> > features as well.
> > >> > > > >> >
> > >> > > > >> > It is wire compatible with 0.92.x. 0.92 clients should work
> > >> with
> > >> > > 0.94
> > >> > > > >> > servers and vice versa.
> > >> > > > >> >
> > >> > > > >> > You can do a rolling restart to get your 0.92.x HBase up on
> > >> this
> > >> > > > >> 0.94.0RC.
> > >> > > > >> >
> > >> > > > >> > The full list of changes is available here:
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12316419
> > >> > > > >> >
> > >> > > > >> > Please take this RC for a spin, check out the doc, etc, and
> > >> vote
> > >> > > +1/-1
> > >> > > > >> by
> > >> > > > >> > May 8th on whether we should release this as 0.94.0.
> > >> > > > >> >
> > >> > > > >> > Thanks.
> > >> > > > >> >
> > >> > > > >> > -- Lars
> > >> > > > >> >
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > > > >> --
> > >> > > > >> Todd Lipcon
> > >> > > > >> Software Engineer, Cloudera
> > >> > > > >>
> > >> > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

--f46d043c81ecf58d2e04bf3d3c1d--