hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: Second release candidate for HBase 0.99.0 (RC1) is available. Please vote by 09/19/2014
Date Sun, 21 Sep 2014 11:48:21 GMT
Sorry, was HTML formated.
0.98.3 => 0,25
0.98.4 => 0,25
0.98.5 => 67,34
0.98.6 => 67,49
0.99.0 => 0,63

On my todo:
1) Get updated numbers from 0.99.0
2) Take a look at the test code to make sure it still test the same thing.
3) Re-run 0.98.4 and 0.98.5


JM

2014-09-20 23:01 GMT-04:00 Ted Yu <yuzhihong@gmail.com>:

> J-M:
> Can you reformat the numbers below - it is hard to read.
> Maybe put them in multiple rows.
>
> On Sat, Sep 20, 2014 at 7:55 PM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
> > I looked a bit closer to the numbers. Just for FilteredScanTest for now.
> >
> > Here are the results for the few last versions. Is it possible that
> > something went into 0.98.5 but not into 0.99?
> >
> > 0.98.30.98.40.98.50.98.60.99.00,250,2567,3467,490,63
> > I will re-run for 0.98.4 and 0.98.6. Numbers for 0.99.0 are already from
> my
> > 2nd run.
> >
> > JM
> >
> > 2014-09-20 14:28 GMT-04:00 Jean-Marc Spaggiari <jean-marc@spaggiari.org
> >:
> >
> > > Sure. I will. I will also update the numbers as soon as I have the new
> > > one. Will also run some other perfs tests to compare with 0.98.
> > >
> > > JM
> > >
> > > 2014-09-20 14:02 GMT-04:00 Enis Söztutar <enis.soz@gmail.com>:
> > >
> > > Agreed. Will be good to fix it in 0.99.1.
> > >>
> > >> On Saturday, September 20, 2014, Stack <stack@duboce.net> wrote:
> > >>
> > >> > JMS file blocker against 0.99.1?
> > >> > Good work boss
> > >> > Stack
> > >> > On Sep 19, 2014 7:00 PM, "Jean-Marc Spaggiari" <
> > jean-marc@spaggiari.org
> > >> > <javascript:;>>
> > >> > wrote:
> > >> >
> > >> > > Sorry, here is the destination:
> > >> > >
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://docs.google.com/spreadsheets/d/1yo-A-f4tjchdT9R-hkh6CkcXrbBHG_K2Y_ptF9QPT1Q/edit?usp=sharing
> > >> > >
> > >> > > 2014-09-19 21:55 GMT-04:00 Andrew Purtell <apurtell@apache.org
> > >> > <javascript:;>>:
> > >> > >
> > >> > > > On Fri, Sep 19, 2014 at 6:44 PM, Jean-Marc Spaggiari
> > >> > > > <jean-marc@spaggiari.org <javascript:;>> wrote:
> > >> > > > > Results are there: http://tiny.cloudera.com/sjeyn
> > >> > > >
> > >> > > > I get "Access not possible"
> > >> > > >
> > >> > > > On Fri, Sep 19, 2014 at 6:44 PM, Jean-Marc Spaggiari
> > >> > > > <jean-marc@spaggiari.org <javascript:;>> wrote:
> > >> > > > > UniformRandomReadBenchmark is not working anymore:
> > >> > > > >
> > >> > > > > 2014-09-19 05:18:54,719 INFO  [0]
> > >> hbase.HFilePerformanceEvaluation:
> > >> > > > Running
> > >> > > > > UniformRandomReadBenchmark for 1000000 rows.
> > >> > > > > 2014-09-19 05:18:54,719 INFO  [0] hfile.CacheConfig:
> > >> > > CacheConfig:disabled
> > >> > > > > Exception in thread "0" java.lang.AssertionError: Expected
> > >> 0000472128
> > >> > > but
> > >> > > > > got 0000472127
> > >> > > > >         at
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:50)
> > >> > > > >         at
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:45)
> > >> > > > >         at
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.hbase.HFilePerformanceEvaluation$UniformRandomReadBenchmark.doRow(HFilePerformanceEvaluation.java:295)
> > >> > > > >         at
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.hbase.HFilePerformanceEvaluation$RowOrientedBenchmark.run(HFilePerformanceEvaluation.java:169)
> > >> > > > >         at
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.hbase.HFilePerformanceEvaluation.runBenchmark(HFilePerformanceEvaluation.java:121)
> > >> > > > >         at
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> org.apache.hadoop.hbase.HFilePerformanceEvaluation$2.run(HFilePerformanceEvaluation.java:87)
> > >> > > > >         at java.lang.Thread.run(Thread.java:744)
> > >> > > > >
> > >> > > > > Also, on the performance side, I got very strange results.
> > >> > > > >
> > >> > > > > Some tests are as fast as 0.98.6, some are WAY slower...
> > >> > > > >
> > >> > > > > Results are there: http://tiny.cloudera.com/sjeyn I
will
> > restart
> > >> the
> > >> > > > tests
> > >> > > > > tomorrow and get new numbers by mid next week.
> > >> > > > >
> > >> > > > > Appart from that, it works. And since it's a dev release,
I'm
> +1
> > >> with
> > >> > > it.
> > >> > > > >
> > >> > > > > JM
> > >> > > > >
> > >> > > > > 2014-09-19 21:28 GMT-04:00 Andrew Purtell <
> apurtell@apache.org
> > >> > <javascript:;>>:
> > >> > > > >
> > >> > > > >> +1
> > >> > > > >>
> > >> > > > >> Checked file sums and signature
> > >> > > > >> Unpacked tarball, looked at directory layout
> > >> > > > >> Started up a minicluster in tree, did a quick ingest
test, it
> > >> > > succeeded
> > >> > > > >>
> > >> > > > >>
> > >> > > > >> On Tue, Sep 16, 2014 at 1:52 PM, Enis Söztutar
<
> > enis@apache.org
> > >> > <javascript:;>>
> > >> > > wrote:
> > >> > > > >> > I am very happy to say that the second HBase
0.99.0 release
> > >> > > candidate
> > >> > > > >> (RC1)
> > >> > > > >> > is
> > >> > > > >> > available for download at
> > >> > > > >> https://people.apache.org/~enis/hbase-0.99.0RC1/
> > >> > > > >> >
> > >> > > > >> > Maven artifacts are also available in the
temporary
> > repository
> > >> > > > >> >
> > >> > > >
> > >> >
> > https://repository.apache.org/content/repositories/orgapachehbase-1039/
> > >> > > > >> >
> > >> > > > >> > Signed with my code signing key E964B5FF.
Can be found
> here:
> > >> > > > >> > https://people.apache.org/keys/committer/enis.asc
> > >> > > > >> >
> > >> > > > >> > Signed tag in the repository can be found
here:
> > >> > > > >> >
> > >> > > > >>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=336a63b5f58bb1c4859e3c2b6a2f5c0d68a7d9e1
> > >> > > > >> >
> > >> > > > >> > NOTE IN GIGANTIC LETTERS THAT THIS IS A DEVELOPER
RELEASE.
> > >> > > > >> > DO NOT USE THIS RELEASE IN PRODUCTION.
> > >> > > > >> >
> > >> > > > >> > HBase 0.99.0 is a "developer preview" release,
and an
> > >> odd-numbered
> > >> > > > >> release
> > >> > > > >> > as
> > >> > > > >> > defined in
> > >> > > > >>
> https://hbase.apache.org/book/upgrading.html#hbase.versioning.
> > >> > > > >> > This release IS NOT intended for production
use, and does
> not
> > >> > > contain
> > >> > > > any
> > >> > > > >> > backwards or forwards compatibility guarantees
(even within
> > >> minor
> > >> > > > >> versions
> > >> > > > >> > 0.99.x). Please refrain from deploying this
over important
> > >> data.
> > >> > > > >> >
> > >> > > > >> > 0.99.0 release is provided from branch-1,
which will be the
> > >> basis
> > >> > > for
> > >> > > > >> > HBase-1.0
> > >> > > > >> > release. A few 0.99.x releases are planned
before 1.0. The
> > >> reason
> > >> > > for
> > >> > > > >> > doing a developer preview release is to get
more testing
> for
> > >> the
> > >> > > > branch-1
> > >> > > > >> > code
> > >> > > > >> > that will be released soon as HBase-1.0.0.
Thus, all
> > >> contribution
> > >> > in
> > >> > > > >> terms
> > >> > > > >> > of
> > >> > > > >> > testing, benchmarking, checking API / source
/wire
> > >> compatibility,
> > >> > > > >> checking
> > >> > > > >> > out
> > >> > > > >> > documentation and further code contribution
is highly
> > >> appreciated.
> > >> > > 1.0
> > >> > > > >> will
> > >> > > > >> > be
> > >> > > > >> > the first series in the 1.x line of releases
which are
> > >> expected to
> > >> > > > keep
> > >> > > > >> > compatibility with previous 1.x releases.
Thus it is very
> > >> > important
> > >> > > to
> > >> > > > >> > check
> > >> > > > >> > the client side and server side APIs for compatibility
and
> > >> > > > >> maintainability
> > >> > > > >> > concerns for future releases.
> > >> > > > >> >
> > >> > > > >> > 0.99.0 contains slightly more than 1K issues
resolved with
> > many
> > >> > > > >> > improvements
> > >> > > > >> > and bug fixes. The theme of (eventual) 1.0
release is to
> > >> become a
> > >> > > > stable
> > >> > > > >> > base
> > >> > > > >> > for future 1.x series of releases. 1.0 release
will aim to
> > >> achieve
> > >> > > at
> > >> > > > >> least
> > >> > > > >> > the same level of stability of 0.98 releases
without
> > >> introducing
> > >> > too
> > >> > > > many
> > >> > > > >> > new
> > >> > > > >> > features.
> > >> > > > >> >
> > >> > > > >> > Some work has been under way to clearly mark
and
> > differentiate
> > >> > > client
> > >> > > > >> > facing
> > >> > > > >> > APIs, and redefine some of the client interfaces
for
> > improving
> > >> > > > semantics,
> > >> > > > >> > easy
> > >> > > > >> > of use and maintainability. 0.99.0 contains
some of the
> > >> completed
> > >> > > > items,
> > >> > > > >> > and
> > >> > > > >> > the rest can be found HBASE-10602. Marking/remarking
of
> > >> interfaces
> > >> > > > with
> > >> > > > >> > InterfaceAudience has also been going on,
which will
> identify
> > >> > areas
> > >> > > > for
> > >> > > > >> > compatibility (with clients, coprocessors
and dependent
> > >> projects
> > >> > > like
> > >> > > > >> > Phoenix)
> > >> > > > >> > for future releases.
> > >> > > > >> >
> > >> > > > >> > Starting with 0.99.0, the HBase master server
and backup
> > master
> > >> > > > servers
> > >> > > > >> > will
> > >> > > > >> > also act as a region server. RPC port and
info port for web
> > UI
> > >> is
> > >> > > > shared
> > >> > > > >> > for
> > >> > > > >> > the master and region server roles. Active
master and
> backup
> > >> > masters
> > >> > > > by
> > >> > > > >> > default
> > >> > > > >> > will NOT be hosting any regions unless configured
> otherwise.
> > >> See
> > >> > > > >> > HBASE-10569
> > >> > > > >> > and HBASE-11604 for more details.
> > >> > > > >> >
> > >> > > > >> > This release contains Phase 1 items for experimental
"Read
> > >> > > > availability
> > >> > > > >> > using
> > >> > > > >> > timeline consistent region replicas" feature.
A region can
> be
> > >> > hosted
> > >> > > > in
> > >> > > > >> > multiple region servers in read-only mode.
One of the
> > replicas
> > >> for
> > >> > > the
> > >> > > > >> > region
> > >> > > > >> > will be primary, accepting writes, and other
replicas will
> be
> > >> > > sharing
> > >> > > > the
> > >> > > > >> > same
> > >> > > > >> > data files. Read requests can be done against
any replica
> for
> > >> the
> > >> > > > region
> > >> > > > >> > with
> > >> > > > >> > backup RPCs for high availability with timeline
consistency
> > >> > > > guarantees.
> > >> > > > >> > More
> > >> > > > >> > information can be found at HBASE-10070.
> > >> > > > >> >
> > >> > > > >> > Other notable improvements in this release
are
> > >> > > > >> >  - Automatic tuning of global memstore and
block cache
> sizes
> > >> > > > >> >  - Various security, tags and visibility labels
> improvements
> > >> > > > >> >  - Bucket cache improvements (usability and
compressed data
> > >> > blocks)
> > >> > > > >> >  - A new pluggable replication endpoint to
plug in to
> HBase's
> > >> > > > >> inter-cluster
> > >> > > > >> >    replication to replicate to a custom data
store
> > >> > > > >> >  - A Dockerfile to easily build and run HBase
from source
> > >> > > > >> >  - Internal refactoring for abstracting away
zookeeper
> usage
> > >> > > > >> >  - Truncate table command
> > >> > > > >> >  - Region assignment to use hbase:meta table
instead of
> > >> zookeeper
> > >> > > for
> > >> > > > >> faster
> > >> > > > >> >    region assignment (disabled by default)
> > >> > > > >> >  - Better support for Cell interface internally
in read and
> > >> write
> > >> > > > paths
> > >> > > > >> for
> > >> > > > >> >    better performance and flexibility
> > >> > > > >> >  - Combining internal mvcc and seqId infrastructure
> > >> > > > >> >  - Extensive documentation improvements
> > >> > > > >> >  - Numerous improvements in other areas and
bug fixes.
> > >> > > > >> >
> > >> > > > >> > The release has these changes in default behaviour:
> > >> > > > >> >  - hfile version 3 has been enabled by default
> > >> > > > >> >  - Distributed log replay has been enabled
by default
> > >> > > > >> >  - Slab cache has been removed (use bucket
cache instead)
> > >> > > > >> >  - Some client facing APIs (HTableInterface,
etc) has been
> > >> > > deprecated
> > >> > > > and
> > >> > > > >> >    replaced.
> > >> > > > >> >
> > >> > > > >> > We have also adopted an orca as the mascot
for Apache
> HBase.
> > >> See
> > >> > it
> > >> > > at
> > >> > > > >> > https://hbase.apache.org/
> > >> > > > >> >
> > >> > > > >> > Thanks for everybody who have contributed
to this release.
> > Full
> > >> > list
> > >> > > > of
> > >> > > > >> the
> > >> > > > >> > issues
> > >> > > > >> > can be found here:
> > >> > > > >> >
> > >> > > > >>
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325675
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >> > Compatibility
> > >> > > > >> > -------------
> > >> > > > >> > 0.99.x series of releases are preview of upcoming
1.0
> > release.
> > >> > > > HBase-1.0
> > >> > > > >> is
> > >> > > > >> > planned to be wire compatible with 0.98.x
and 0.96.x
> > releases.
> > >> > > Clients
> > >> > > > >> and
> > >> > > > >> > servers running in different versions as long
as new
> features
> > >> are
> > >> > > not
> > >> > > > >> used
> > >> > > > >> > should be possible.
> > >> > > > >> > A rolling upgrade from 0.98.x clusters to
0.99.0 is
> supported
> > >> as
> > >> > > well.
> > >> > > > >> > 0.99.0
> > >> > > > >> > introduces a new file format (hfile v3) that
is enabled by
> > >> default
> > >> > > > that
> > >> > > > >> > 0.96.x code cannot read. Rolling upgrade from
0.96 directly
> > to
> > >> > 0.99
> > >> > > is
> > >> > > > >> not
> > >> > > > >> > tested.
> > >> > > > >> >
> > >> > > > >> > Direct migration from 0.94.x is also supported
but it IS
> NOT
> > >> > tested.
> > >> > > > >> Before
> > >> > > > >> > HBase-1.0 we expect to complete the test.
> > >> > > > >> >
> > >> > > > >> > Binary compatibility at the Java API layer
with earlier
> > >> versions
> > >> > > > (0.98.x,
> > >> > > > >> > 0.96.x and 0.94.x) is not supported. You may
have to
> > recompile
> > >> > your
> > >> > > > >> client
> > >> > > > >> > code and any server side code (coprocessors,
filters etc)
> > >> > referring
> > >> > > to
> > >> > > > >> > hbase jars.
> > >> > > > >> >
> > >> > > > >> > 0.99.0 release introduces new APIs, and deprecates
some of
> > >> > > > commonly-used
> > >> > > > >> > client side APIs (HTableInterface, etc). However,
client
> side
> > >> code
> > >> > > is
> > >> > > > >> > expected
> > >> > > > >> > to be source compatible with earlier versions.
However we
> > >> advise
> > >> > to
> > >> > > > >> update
> > >> > > > >> > your application to use the new style of APIs,
since
> > deprecated
> > >> > APIs
> > >> > > > >> might
> > >> > > > >> > be removed in future releases (2.x).
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >> > Supported Hadoop versions
> > >> > > > >> > -------------------------
> > >> > > > >> > 0.99.0 release drops support for Hadoop-1.x
releases. Only
> > >> > > Hadoop-2.x
> > >> > > > >> > releases are supported. Hadoop-2.4.x and Hadoop-2.5.x
> > releases
> > >> are
> > >> > > the
> > >> > > > >> > most tested hadoop releases and we recommend
running with
> > those
> > >> > > > versions.
> > >> > > > >> > Earlier Hadoop-2 based releases (hadoop-2.2.x
and 2.3.x)
> are
> > >> not
> > >> > > > tested
> > >> > > > >> to
> > >> > > > >> > the full
> > >> > > > >> > extend. More information can be found here:
> > >> > > > >> > https://hbase.apache.org/book/configuration.html#hadoop
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >> > Supported Java versions
> > >> > > > >> > -------------------------
> > >> > > > >> > 0.99.0 release drops support for JDK6. Only
JDK7 is
> > supported.
> > >> > JDK8
> > >> > > > >> support
> > >> > > > >> > is experimental. More information can be found
here:
> > >> > > > >> > https://hbase.apache.org/book/configuration.html#java
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >> > Voting
> > >> > > > >> > ------
> > >> > > > >> > Please try to test and vote on this release
by 09/19/2014
> > >> 11:59PM
> > >> > > PDT.
> > >> > > > >> > This is a developer (beta) release, so we
have a shorter
> vote
> > >> > time.
> > >> > > > >> >
> > >> > > > >> > [] +1 Release the artifacts as 0.99.0
> > >> > > > >> > [] -1 DO NOT release the artifacts as 0.99.0,
because...
> > >> > > > >> >
> > >> > > > >> > Thanks for helping to get a stable release
out!
> > >> > > > >> >
> > >> > > > >> > Enis
> > >> > > > >>
> > >> > > > >>
> > >> > > > >>
> > >> > > > >> --
> > >> > > > >> Best regards,
> > >> > > > >>
> > >> > > > >>    - Andy
> > >> > > > >>
> > >> > > > >> Problems worthy of attack prove their worth by
hitting back.
> -
> > >> Piet
> > >> > > > >> Hein (via Tom White)
> > >> > > > >>
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > --
> > >> > > > Best regards,
> > >> > > >
> > >> > > >    - Andy
> > >> > > >
> > >> > > > Problems worthy of attack prove their worth by hitting back.
-
> > Piet
> > >> > > > Hein (via Tom White)
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message