hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: Second release candidate for HBase 0.99.0 (RC1) is available. Please vote by 09/19/2014
Date Sun, 21 Sep 2014 12:27:40 GMT
1) Confirmed. 2nd run gave similar results.
2) Checked the logs and so far, nothing wrong. Looking at the
FilteredScanTest code now.

Opened HBASE-12040 and HBASE-12042

2014-09-21 7:48 GMT-04:00 Jean-Marc Spaggiari <jean-marc@spaggiari.org>:

> Sorry, was HTML formated.
> 0.98.3 => 0,25
> 0.98.4 => 0,25
> 0.98.5 => 67,34
> 0.98.6 => 67,49
> 0.99.0 => 0,63
>
> On my todo:
> 1) Get updated numbers from 0.99.0
> 2) Take a look at the test code to make sure it still test the same thing.
> 3) Re-run 0.98.4 and 0.98.5
>
>
> JM
>
> 2014-09-20 23:01 GMT-04:00 Ted Yu <yuzhihong@gmail.com>:
>
> J-M:
>> Can you reformat the numbers below - it is hard to read.
>> Maybe put them in multiple rows.
>>
>> On Sat, Sep 20, 2014 at 7:55 PM, Jean-Marc Spaggiari <
>> jean-marc@spaggiari.org> wrote:
>>
>> > I looked a bit closer to the numbers. Just for FilteredScanTest for now.
>> >
>> > Here are the results for the few last versions. Is it possible that
>> > something went into 0.98.5 but not into 0.99?
>> >
>> > 0.98.30.98.40.98.50.98.60.99.00,250,2567,3467,490,63
>> > I will re-run for 0.98.4 and 0.98.6. Numbers for 0.99.0 are already
>> from my
>> > 2nd run.
>> >
>> > JM
>> >
>> > 2014-09-20 14:28 GMT-04:00 Jean-Marc Spaggiari <jean-marc@spaggiari.org
>> >:
>> >
>> > > Sure. I will. I will also update the numbers as soon as I have the new
>> > > one. Will also run some other perfs tests to compare with 0.98.
>> > >
>> > > JM
>> > >
>> > > 2014-09-20 14:02 GMT-04:00 Enis Söztutar <enis.soz@gmail.com>:
>> > >
>> > > Agreed. Will be good to fix it in 0.99.1.
>> > >>
>> > >> On Saturday, September 20, 2014, Stack <stack@duboce.net> wrote:
>> > >>
>> > >> > JMS file blocker against 0.99.1?
>> > >> > Good work boss
>> > >> > Stack
>> > >> > On Sep 19, 2014 7:00 PM, "Jean-Marc Spaggiari" <
>> > jean-marc@spaggiari.org
>> > >> > <javascript:;>>
>> > >> > wrote:
>> > >> >
>> > >> > > Sorry, here is the destination:
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> https://docs.google.com/spreadsheets/d/1yo-A-f4tjchdT9R-hkh6CkcXrbBHG_K2Y_ptF9QPT1Q/edit?usp=sharing
>> > >> > >
>> > >> > > 2014-09-19 21:55 GMT-04:00 Andrew Purtell <apurtell@apache.org
>> > >> > <javascript:;>>:
>> > >> > >
>> > >> > > > On Fri, Sep 19, 2014 at 6:44 PM, Jean-Marc Spaggiari
>> > >> > > > <jean-marc@spaggiari.org <javascript:;>>
wrote:
>> > >> > > > > Results are there: http://tiny.cloudera.com/sjeyn
>> > >> > > >
>> > >> > > > I get "Access not possible"
>> > >> > > >
>> > >> > > > On Fri, Sep 19, 2014 at 6:44 PM, Jean-Marc Spaggiari
>> > >> > > > <jean-marc@spaggiari.org <javascript:;>>
wrote:
>> > >> > > > > UniformRandomReadBenchmark is not working anymore:
>> > >> > > > >
>> > >> > > > > 2014-09-19 05:18:54,719 INFO  [0]
>> > >> hbase.HFilePerformanceEvaluation:
>> > >> > > > Running
>> > >> > > > > UniformRandomReadBenchmark for 1000000 rows.
>> > >> > > > > 2014-09-19 05:18:54,719 INFO  [0] hfile.CacheConfig:
>> > >> > > CacheConfig:disabled
>> > >> > > > > Exception in thread "0" java.lang.AssertionError:
Expected
>> > >> 0000472128
>> > >> > > but
>> > >> > > > > got 0000472127
>> > >> > > > >         at
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:50)
>> > >> > > > >         at
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:45)
>> > >> > > > >         at
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> org.apache.hadoop.hbase.HFilePerformanceEvaluation$UniformRandomReadBenchmark.doRow(HFilePerformanceEvaluation.java:295)
>> > >> > > > >         at
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> org.apache.hadoop.hbase.HFilePerformanceEvaluation$RowOrientedBenchmark.run(HFilePerformanceEvaluation.java:169)
>> > >> > > > >         at
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> org.apache.hadoop.hbase.HFilePerformanceEvaluation.runBenchmark(HFilePerformanceEvaluation.java:121)
>> > >> > > > >         at
>> > >> > > > >
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> org.apache.hadoop.hbase.HFilePerformanceEvaluation$2.run(HFilePerformanceEvaluation.java:87)
>> > >> > > > >         at java.lang.Thread.run(Thread.java:744)
>> > >> > > > >
>> > >> > > > > Also, on the performance side, I got very strange
results.
>> > >> > > > >
>> > >> > > > > Some tests are as fast as 0.98.6, some are WAY
slower...
>> > >> > > > >
>> > >> > > > > Results are there: http://tiny.cloudera.com/sjeyn
I will
>> > restart
>> > >> the
>> > >> > > > tests
>> > >> > > > > tomorrow and get new numbers by mid next week.
>> > >> > > > >
>> > >> > > > > Appart from that, it works. And since it's a dev
release,
>> I'm +1
>> > >> with
>> > >> > > it.
>> > >> > > > >
>> > >> > > > > JM
>> > >> > > > >
>> > >> > > > > 2014-09-19 21:28 GMT-04:00 Andrew Purtell <
>> apurtell@apache.org
>> > >> > <javascript:;>>:
>> > >> > > > >
>> > >> > > > >> +1
>> > >> > > > >>
>> > >> > > > >> Checked file sums and signature
>> > >> > > > >> Unpacked tarball, looked at directory layout
>> > >> > > > >> Started up a minicluster in tree, did a quick
ingest test,
>> it
>> > >> > > succeeded
>> > >> > > > >>
>> > >> > > > >>
>> > >> > > > >> On Tue, Sep 16, 2014 at 1:52 PM, Enis Söztutar
<
>> > enis@apache.org
>> > >> > <javascript:;>>
>> > >> > > wrote:
>> > >> > > > >> > I am very happy to say that the second
HBase 0.99.0
>> release
>> > >> > > candidate
>> > >> > > > >> (RC1)
>> > >> > > > >> > is
>> > >> > > > >> > available for download at
>> > >> > > > >> https://people.apache.org/~enis/hbase-0.99.0RC1/
>> > >> > > > >> >
>> > >> > > > >> > Maven artifacts are also available in
the temporary
>> > repository
>> > >> > > > >> >
>> > >> > > >
>> > >> >
>> > https://repository.apache.org/content/repositories/orgapachehbase-1039/
>> > >> > > > >> >
>> > >> > > > >> > Signed with my code signing key E964B5FF.
Can be found
>> here:
>> > >> > > > >> > https://people.apache.org/keys/committer/enis.asc
>> > >> > > > >> >
>> > >> > > > >> > Signed tag in the repository can be found
here:
>> > >> > > > >> >
>> > >> > > > >>
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=336a63b5f58bb1c4859e3c2b6a2f5c0d68a7d9e1
>> > >> > > > >> >
>> > >> > > > >> > NOTE IN GIGANTIC LETTERS THAT THIS IS
A DEVELOPER RELEASE.
>> > >> > > > >> > DO NOT USE THIS RELEASE IN PRODUCTION.
>> > >> > > > >> >
>> > >> > > > >> > HBase 0.99.0 is a "developer preview"
release, and an
>> > >> odd-numbered
>> > >> > > > >> release
>> > >> > > > >> > as
>> > >> > > > >> > defined in
>> > >> > > > >>
>> https://hbase.apache.org/book/upgrading.html#hbase.versioning.
>> > >> > > > >> > This release IS NOT intended for production
use, and does
>> not
>> > >> > > contain
>> > >> > > > any
>> > >> > > > >> > backwards or forwards compatibility guarantees
(even
>> within
>> > >> minor
>> > >> > > > >> versions
>> > >> > > > >> > 0.99.x). Please refrain from deploying
this over important
>> > >> data.
>> > >> > > > >> >
>> > >> > > > >> > 0.99.0 release is provided from branch-1,
which will be
>> the
>> > >> basis
>> > >> > > for
>> > >> > > > >> > HBase-1.0
>> > >> > > > >> > release. A few 0.99.x releases are planned
before 1.0. The
>> > >> reason
>> > >> > > for
>> > >> > > > >> > doing a developer preview release is to
get more testing
>> for
>> > >> the
>> > >> > > > branch-1
>> > >> > > > >> > code
>> > >> > > > >> > that will be released soon as HBase-1.0.0.
Thus, all
>> > >> contribution
>> > >> > in
>> > >> > > > >> terms
>> > >> > > > >> > of
>> > >> > > > >> > testing, benchmarking, checking API /
source /wire
>> > >> compatibility,
>> > >> > > > >> checking
>> > >> > > > >> > out
>> > >> > > > >> > documentation and further code contribution
is highly
>> > >> appreciated.
>> > >> > > 1.0
>> > >> > > > >> will
>> > >> > > > >> > be
>> > >> > > > >> > the first series in the 1.x line of releases
which are
>> > >> expected to
>> > >> > > > keep
>> > >> > > > >> > compatibility with previous 1.x releases.
Thus it is very
>> > >> > important
>> > >> > > to
>> > >> > > > >> > check
>> > >> > > > >> > the client side and server side APIs for
compatibility and
>> > >> > > > >> maintainability
>> > >> > > > >> > concerns for future releases.
>> > >> > > > >> >
>> > >> > > > >> > 0.99.0 contains slightly more than 1K
issues resolved with
>> > many
>> > >> > > > >> > improvements
>> > >> > > > >> > and bug fixes. The theme of (eventual)
1.0 release is to
>> > >> become a
>> > >> > > > stable
>> > >> > > > >> > base
>> > >> > > > >> > for future 1.x series of releases. 1.0
release will aim to
>> > >> achieve
>> > >> > > at
>> > >> > > > >> least
>> > >> > > > >> > the same level of stability of 0.98 releases
without
>> > >> introducing
>> > >> > too
>> > >> > > > many
>> > >> > > > >> > new
>> > >> > > > >> > features.
>> > >> > > > >> >
>> > >> > > > >> > Some work has been under way to clearly
mark and
>> > differentiate
>> > >> > > client
>> > >> > > > >> > facing
>> > >> > > > >> > APIs, and redefine some of the client
interfaces for
>> > improving
>> > >> > > > semantics,
>> > >> > > > >> > easy
>> > >> > > > >> > of use and maintainability. 0.99.0 contains
some of the
>> > >> completed
>> > >> > > > items,
>> > >> > > > >> > and
>> > >> > > > >> > the rest can be found HBASE-10602. Marking/remarking
of
>> > >> interfaces
>> > >> > > > with
>> > >> > > > >> > InterfaceAudience has also been going
on, which will
>> identify
>> > >> > areas
>> > >> > > > for
>> > >> > > > >> > compatibility (with clients, coprocessors
and dependent
>> > >> projects
>> > >> > > like
>> > >> > > > >> > Phoenix)
>> > >> > > > >> > for future releases.
>> > >> > > > >> >
>> > >> > > > >> > Starting with 0.99.0, the HBase master
server and backup
>> > master
>> > >> > > > servers
>> > >> > > > >> > will
>> > >> > > > >> > also act as a region server. RPC port
and info port for
>> web
>> > UI
>> > >> is
>> > >> > > > shared
>> > >> > > > >> > for
>> > >> > > > >> > the master and region server roles. Active
master and
>> backup
>> > >> > masters
>> > >> > > > by
>> > >> > > > >> > default
>> > >> > > > >> > will NOT be hosting any regions unless
configured
>> otherwise.
>> > >> See
>> > >> > > > >> > HBASE-10569
>> > >> > > > >> > and HBASE-11604 for more details.
>> > >> > > > >> >
>> > >> > > > >> > This release contains Phase 1 items for
experimental "Read
>> > >> > > > availability
>> > >> > > > >> > using
>> > >> > > > >> > timeline consistent region replicas" feature.
A region
>> can be
>> > >> > hosted
>> > >> > > > in
>> > >> > > > >> > multiple region servers in read-only mode.
One of the
>> > replicas
>> > >> for
>> > >> > > the
>> > >> > > > >> > region
>> > >> > > > >> > will be primary, accepting writes, and
other replicas
>> will be
>> > >> > > sharing
>> > >> > > > the
>> > >> > > > >> > same
>> > >> > > > >> > data files. Read requests can be done
against any replica
>> for
>> > >> the
>> > >> > > > region
>> > >> > > > >> > with
>> > >> > > > >> > backup RPCs for high availability with
timeline
>> consistency
>> > >> > > > guarantees.
>> > >> > > > >> > More
>> > >> > > > >> > information can be found at HBASE-10070.
>> > >> > > > >> >
>> > >> > > > >> > Other notable improvements in this release
are
>> > >> > > > >> >  - Automatic tuning of global memstore
and block cache
>> sizes
>> > >> > > > >> >  - Various security, tags and visibility
labels
>> improvements
>> > >> > > > >> >  - Bucket cache improvements (usability
and compressed
>> data
>> > >> > blocks)
>> > >> > > > >> >  - A new pluggable replication endpoint
to plug in to
>> HBase's
>> > >> > > > >> inter-cluster
>> > >> > > > >> >    replication to replicate to a custom
data store
>> > >> > > > >> >  - A Dockerfile to easily build and run
HBase from source
>> > >> > > > >> >  - Internal refactoring for abstracting
away zookeeper
>> usage
>> > >> > > > >> >  - Truncate table command
>> > >> > > > >> >  - Region assignment to use hbase:meta
table instead of
>> > >> zookeeper
>> > >> > > for
>> > >> > > > >> faster
>> > >> > > > >> >    region assignment (disabled by default)
>> > >> > > > >> >  - Better support for Cell interface internally
in read
>> and
>> > >> write
>> > >> > > > paths
>> > >> > > > >> for
>> > >> > > > >> >    better performance and flexibility
>> > >> > > > >> >  - Combining internal mvcc and seqId infrastructure
>> > >> > > > >> >  - Extensive documentation improvements
>> > >> > > > >> >  - Numerous improvements in other areas
and bug fixes.
>> > >> > > > >> >
>> > >> > > > >> > The release has these changes in default
behaviour:
>> > >> > > > >> >  - hfile version 3 has been enabled by
default
>> > >> > > > >> >  - Distributed log replay has been enabled
by default
>> > >> > > > >> >  - Slab cache has been removed (use bucket
cache instead)
>> > >> > > > >> >  - Some client facing APIs (HTableInterface,
etc) has been
>> > >> > > deprecated
>> > >> > > > and
>> > >> > > > >> >    replaced.
>> > >> > > > >> >
>> > >> > > > >> > We have also adopted an orca as the mascot
for Apache
>> HBase.
>> > >> See
>> > >> > it
>> > >> > > at
>> > >> > > > >> > https://hbase.apache.org/
>> > >> > > > >> >
>> > >> > > > >> > Thanks for everybody who have contributed
to this release.
>> > Full
>> > >> > list
>> > >> > > > of
>> > >> > > > >> the
>> > >> > > > >> > issues
>> > >> > > > >> > can be found here:
>> > >> > > > >> >
>> > >> > > > >>
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> >
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325675
>> > >> > > > >> >
>> > >> > > > >> >
>> > >> > > > >> > Compatibility
>> > >> > > > >> > -------------
>> > >> > > > >> > 0.99.x series of releases are preview
of upcoming 1.0
>> > release.
>> > >> > > > HBase-1.0
>> > >> > > > >> is
>> > >> > > > >> > planned to be wire compatible with 0.98.x
and 0.96.x
>> > releases.
>> > >> > > Clients
>> > >> > > > >> and
>> > >> > > > >> > servers running in different versions
as long as new
>> features
>> > >> are
>> > >> > > not
>> > >> > > > >> used
>> > >> > > > >> > should be possible.
>> > >> > > > >> > A rolling upgrade from 0.98.x clusters
to 0.99.0 is
>> supported
>> > >> as
>> > >> > > well.
>> > >> > > > >> > 0.99.0
>> > >> > > > >> > introduces a new file format (hfile v3)
that is enabled by
>> > >> default
>> > >> > > > that
>> > >> > > > >> > 0.96.x code cannot read. Rolling upgrade
from 0.96
>> directly
>> > to
>> > >> > 0.99
>> > >> > > is
>> > >> > > > >> not
>> > >> > > > >> > tested.
>> > >> > > > >> >
>> > >> > > > >> > Direct migration from 0.94.x is also supported
but it IS
>> NOT
>> > >> > tested.
>> > >> > > > >> Before
>> > >> > > > >> > HBase-1.0 we expect to complete the test.
>> > >> > > > >> >
>> > >> > > > >> > Binary compatibility at the Java API layer
with earlier
>> > >> versions
>> > >> > > > (0.98.x,
>> > >> > > > >> > 0.96.x and 0.94.x) is not supported. You
may have to
>> > recompile
>> > >> > your
>> > >> > > > >> client
>> > >> > > > >> > code and any server side code (coprocessors,
filters etc)
>> > >> > referring
>> > >> > > to
>> > >> > > > >> > hbase jars.
>> > >> > > > >> >
>> > >> > > > >> > 0.99.0 release introduces new APIs, and
deprecates some of
>> > >> > > > commonly-used
>> > >> > > > >> > client side APIs (HTableInterface, etc).
However, client
>> side
>> > >> code
>> > >> > > is
>> > >> > > > >> > expected
>> > >> > > > >> > to be source compatible with earlier versions.
However we
>> > >> advise
>> > >> > to
>> > >> > > > >> update
>> > >> > > > >> > your application to use the new style
of APIs, since
>> > deprecated
>> > >> > APIs
>> > >> > > > >> might
>> > >> > > > >> > be removed in future releases (2.x).
>> > >> > > > >> >
>> > >> > > > >> >
>> > >> > > > >> > Supported Hadoop versions
>> > >> > > > >> > -------------------------
>> > >> > > > >> > 0.99.0 release drops support for Hadoop-1.x
releases. Only
>> > >> > > Hadoop-2.x
>> > >> > > > >> > releases are supported. Hadoop-2.4.x and
Hadoop-2.5.x
>> > releases
>> > >> are
>> > >> > > the
>> > >> > > > >> > most tested hadoop releases and we recommend
running with
>> > those
>> > >> > > > versions.
>> > >> > > > >> > Earlier Hadoop-2 based releases (hadoop-2.2.x
and 2.3.x)
>> are
>> > >> not
>> > >> > > > tested
>> > >> > > > >> to
>> > >> > > > >> > the full
>> > >> > > > >> > extend. More information can be found
here:
>> > >> > > > >> > https://hbase.apache.org/book/configuration.html#hadoop
>> > >> > > > >> >
>> > >> > > > >> >
>> > >> > > > >> > Supported Java versions
>> > >> > > > >> > -------------------------
>> > >> > > > >> > 0.99.0 release drops support for JDK6.
Only JDK7 is
>> > supported.
>> > >> > JDK8
>> > >> > > > >> support
>> > >> > > > >> > is experimental. More information can
be found here:
>> > >> > > > >> > https://hbase.apache.org/book/configuration.html#java
>> > >> > > > >> >
>> > >> > > > >> >
>> > >> > > > >> > Voting
>> > >> > > > >> > ------
>> > >> > > > >> > Please try to test and vote on this release
by 09/19/2014
>> > >> 11:59PM
>> > >> > > PDT.
>> > >> > > > >> > This is a developer (beta) release, so
we have a shorter
>> vote
>> > >> > time.
>> > >> > > > >> >
>> > >> > > > >> > [] +1 Release the artifacts as 0.99.0
>> > >> > > > >> > [] -1 DO NOT release the artifacts as
0.99.0, because...
>> > >> > > > >> >
>> > >> > > > >> > Thanks for helping to get a stable release
out!
>> > >> > > > >> >
>> > >> > > > >> > Enis
>> > >> > > > >>
>> > >> > > > >>
>> > >> > > > >>
>> > >> > > > >> --
>> > >> > > > >> Best regards,
>> > >> > > > >>
>> > >> > > > >>    - Andy
>> > >> > > > >>
>> > >> > > > >> Problems worthy of attack prove their worth
by hitting
>> back. -
>> > >> Piet
>> > >> > > > >> Hein (via Tom White)
>> > >> > > > >>
>> > >> > > >
>> > >> > > >
>> > >> > > >
>> > >> > > > --
>> > >> > > > Best regards,
>> > >> > > >
>> > >> > > >    - Andy
>> > >> > > >
>> > >> > > > Problems worthy of attack prove their worth by hitting
back. -
>> > Piet
>> > >> > > > Hein (via Tom White)
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message