hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: Second release candidate for HBase 0.99.0 (RC1) is available. Please vote by 09/19/2014
Date Mon, 22 Sep 2014 18:32:15 GMT
Sorry, was busy on another issue.

So.

You were right, this is my "small" cluster. It's 4 nodes. One master, 3 RS.
The "big" cluster (8 nodes) is reserved for Lars (0.94) for now ;)

I run the tests using this command:
for i in {1..10}; do echo; echo -n $i ; /home/hadoop/bin/hadoop fs -rmr
/hbase/*; rm -rf /tmp/*; echo rmr /hbase |
/home/zookeeper/zookeeper-3.4.3/bin/zkCli.sh; bin/start-hbase.sh; sleep 60;
bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 1;
echo balancer | bin/hbase shell; sleep 60; bin/hbase
org.apache.hadoop.hbase.PerformanceEvaluation --rows=100 --nomapred
filterScan 1; bin/stop-hbase.sh; done &>> output.txt

Basically, it removed everything, start HBase, put some data it in, and
settle down. Then run the test. I do that 10 times for each test and remove
the smallest and fastests.

Nodes are 16GB.

Extract from hbase-env.sh:
export JAVA_HOME=/usr/local/jdk1.7.0_45/

# Extra Java CLASSPATH elements.  Optional.
# export HBASE_CLASSPATH=

# The maximum amount of heap to use, in MB. Default is 1000.
export HBASE_HEAPSIZE=10240



Configured propoerties:
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://hbasetest1:9000/hbase</value>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>hbasetest1.distparser.com</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/home/zookeeper</value>
  </property>
  <property>
    <name>fs.default.name</name>
    <value>hdfs://hbasetest1:9000/</value>
  </property>
  <property>
    <name>hbase.regionserver.codecs</name>
    <value>gz</value>
  </property>
  <property>
    <name>ipc.server.tcpnodelay</name>
    <value>true</value>
  </property>
  <property>
    <name>ipc.client.tcpnodelay</name>
    <value>true</value>
  </property>
  <property>
    <name>hbase.regionserver.region.split.policy</name>

<value>org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy</value>
  </property>
  <property>
    <name>hbase.hregion.max.filesize</name>
    <value>1073741824000</value>
  </property>


3 disks only per node:
hbase@hbasetest2:~$ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   1,8T  0 disk
├─sda1   8:1    0  93,1G  0 part /
└─sda2   8:2    0   1,7T  0 part /data1
sdb      8:16   0   1,8T  0 disk
├─sdb1   8:17   0   9,3G  0 part [SWAP]
└─sdb2   8:18   0   1,8T  0 part /data2
sdc      8:32   0   1,8T  0 disk
└─sdc1   8:33   0   1,8T  0 part /data3


data1 to data3 are the datanode partitions.

Drives are pretty empty:
hbase@hbasetest2:~$ df -h
Sys. de fichiers Taille Utilisé Dispo Uti% Monté sur
/dev/sda1           92G     19G   69G  22% /
udev                10M       0   10M   0% /dev
tmpfs              1,6G    272K  1,6G   1% /run
tmpfs              5,0M       0  5,0M   0% /run/lock
tmpfs              5,0G       0  5,0G   0% /run/shm
/dev/sda2          1,8T    399G  1,4T  24% /data1
/dev/sdb2          1,9T    398G  1,4T  22% /data2
/dev/sdc1          1,9T    397G  1,4T  22% /data3

RegionServers are SATA, Master is SSD. Only one ZK server hosted on the
master too.

0.94.x tests run with hadoop 1.2.1.
0.98.x+ tests run with hadoop 2.2.0

I'm trying to build 0.99 from the source to run it and being able to run
some specific revisions. But so far no success (yet) ;)

Just ask me whatever else you might want to know about the cluster. Can
even give you a remote access.

JM

2014-09-22 13:57 GMT-04:00 Stack <stack@duboce.net>:

> JMS, where I get specs on your test cluster and test runs doing perf
> comparision?  I'd like to work on this boss.
> Thanks sir,
> St.Ack
>
> On Mon, Sep 22, 2014 at 10:27 AM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
> > Hey, I agree. Was to enthusiast about this new release that I skipped it
> > was only a dev release ;)
> >
> > 2014-09-22 13:25 GMT-04:00 Andrew Purtell <apurtell@apache.org>:
> >
> > > I think we should not announce developer preview releases on such a
> > > general list. But the occasion of the first 1.0 release should be sent
> > > everywhere possible!
> > >
> > > On Sun, Sep 21, 2014 at 7:44 PM, Enis Söztutar <enis.soz@gmail.com>
> > wrote:
> > > > Do you think is it worth the anounce? The release is not intended for
> > > > general usage but for testing.
> > > >
> > > > Enis
> > > >
> > > > On Sun, Sep 21, 2014 at 4:52 PM, Jean-Marc Spaggiari <
> > > > jean-marc@spaggiari.org> wrote:
> > > >
> > > >> Cool!!! First 0.99 is out!! It's the first stone for the road to
> > 1.0...
> > > >>
> > > >> Enis, any concern if I relay the announcement you will send yo the
> > > Apache
> > > >> Announce mailing list?
> > > >>
> > > >> JM
> > > >> Le 2014-09-21 19:45, "Enis Söztutar" <enis@apache.org> a écrit :
> > > >>
> > > >> > The vote passed with 4 +1s. Although there are known problems in
> > > perf, I
> > > >> > will continue with the release flow since this is a non-production
> > > >> release.
> > > >> > The reason we are doing this release is to encourage more testing
> > and
> > > >> > finding out issues.
> > > >> >
> > > >> > I will send out announcement after 24 hours. Thanks everyone who
> > have
> > > >> > tested and voted.
> > > >> >
> > > >> > Enis
> > > >> >
> > > >> > On Sun, Sep 21, 2014 at 5:27 AM, Jean-Marc Spaggiari <
> > > >> > jean-marc@spaggiari.org> wrote:
> > > >> >
> > > >> > > 1) Confirmed. 2nd run gave similar results.
> > > >> > > 2) Checked the logs and so far, nothing wrong. Looking at the
> > > >> > > FilteredScanTest code now.
> > > >> > >
> > > >> > > Opened HBASE-12040 and HBASE-12042
> > > >> > >
> > > >> > > 2014-09-21 7:48 GMT-04:00 Jean-Marc Spaggiari <
> > > jean-marc@spaggiari.org
> > > >> >:
> > > >> > >
> > > >> > > > Sorry, was HTML formated.
> > > >> > > > 0.98.3 => 0,25
> > > >> > > > 0.98.4 => 0,25
> > > >> > > > 0.98.5 => 67,34
> > > >> > > > 0.98.6 => 67,49
> > > >> > > > 0.99.0 => 0,63
> > > >> > > >
> > > >> > > > On my todo:
> > > >> > > > 1) Get updated numbers from 0.99.0
> > > >> > > > 2) Take a look at the test code to make sure it still test the
> > > same
> > > >> > > thing.
> > > >> > > > 3) Re-run 0.98.4 and 0.98.5
> > > >> > > >
> > > >> > > >
> > > >> > > > JM
> > > >> > > >
> > > >> > > > 2014-09-20 23:01 GMT-04:00 Ted Yu <yuzhihong@gmail.com>:
> > > >> > > >
> > > >> > > > J-M:
> > > >> > > >> Can you reformat the numbers below - it is hard to read.
> > > >> > > >> Maybe put them in multiple rows.
> > > >> > > >>
> > > >> > > >> On Sat, Sep 20, 2014 at 7:55 PM, Jean-Marc Spaggiari <
> > > >> > > >> jean-marc@spaggiari.org> wrote:
> > > >> > > >>
> > > >> > > >> > I looked a bit closer to the numbers. Just for
> > FilteredScanTest
> > > >> for
> > > >> > > now.
> > > >> > > >> >
> > > >> > > >> > Here are the results for the few last versions. Is it
> > possible
> > > >> that
> > > >> > > >> > something went into 0.98.5 but not into 0.99?
> > > >> > > >> >
> > > >> > > >> > 0.98.30.98.40.98.50.98.60.99.00,250,2567,3467,490,63
> > > >> > > >> > I will re-run for 0.98.4 and 0.98.6. Numbers for 0.99.0 are
> > > >> already
> > > >> > > >> from my
> > > >> > > >> > 2nd run.
> > > >> > > >> >
> > > >> > > >> > JM
> > > >> > > >> >
> > > >> > > >> > 2014-09-20 14:28 GMT-04:00 Jean-Marc Spaggiari <
> > > >> > > jean-marc@spaggiari.org
> > > >> > > >> >:
> > > >> > > >> >
> > > >> > > >> > > Sure. I will. I will also update the numbers as soon as I
> > > have
> > > >> the
> > > >> > > new
> > > >> > > >> > > one. Will also run some other perfs tests to compare with
> > > 0.98.
> > > >> > > >> > >
> > > >> > > >> > > JM
> > > >> > > >> > >
> > > >> > > >> > > 2014-09-20 14:02 GMT-04:00 Enis Söztutar <
> > enis.soz@gmail.com
> > > >:
> > > >> > > >> > >
> > > >> > > >> > > Agreed. Will be good to fix it in 0.99.1.
> > > >> > > >> > >>
> > > >> > > >> > >> On Saturday, September 20, 2014, Stack <
> stack@duboce.net>
> > > >> wrote:
> > > >> > > >> > >>
> > > >> > > >> > >> > JMS file blocker against 0.99.1?
> > > >> > > >> > >> > Good work boss
> > > >> > > >> > >> > Stack
> > > >> > > >> > >> > On Sep 19, 2014 7:00 PM, "Jean-Marc Spaggiari" <
> > > >> > > >> > jean-marc@spaggiari.org
> > > >> > > >> > >> > <javascript:;>>
> > > >> > > >> > >> > wrote:
> > > >> > > >> > >> >
> > > >> > > >> > >> > > Sorry, here is the destination:
> > > >> > > >> > >> > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://docs.google.com/spreadsheets/d/1yo-A-f4tjchdT9R-hkh6CkcXrbBHG_K2Y_ptF9QPT1Q/edit?usp=sharing
> > > >> > > >> > >> > >
> > > >> > > >> > >> > > 2014-09-19 21:55 GMT-04:00 Andrew Purtell <
> > > >> > apurtell@apache.org
> > > >> > > >> > >> > <javascript:;>>:
> > > >> > > >> > >> > >
> > > >> > > >> > >> > > > On Fri, Sep 19, 2014 at 6:44 PM, Jean-Marc
> Spaggiari
> > > >> > > >> > >> > > > <jean-marc@spaggiari.org <javascript:;>> wrote:
> > > >> > > >> > >> > > > > Results are there:
> http://tiny.cloudera.com/sjeyn
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > I get "Access not possible"
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > On Fri, Sep 19, 2014 at 6:44 PM, Jean-Marc
> Spaggiari
> > > >> > > >> > >> > > > <jean-marc@spaggiari.org <javascript:;>> wrote:
> > > >> > > >> > >> > > > > UniformRandomReadBenchmark is not working
> anymore:
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > 2014-09-19 05:18:54,719 INFO  [0]
> > > >> > > >> > >> hbase.HFilePerformanceEvaluation:
> > > >> > > >> > >> > > > Running
> > > >> > > >> > >> > > > > UniformRandomReadBenchmark for 1000000 rows.
> > > >> > > >> > >> > > > > 2014-09-19 05:18:54,719 INFO  [0]
> > hfile.CacheConfig:
> > > >> > > >> > >> > > CacheConfig:disabled
> > > >> > > >> > >> > > > > Exception in thread "0"
> java.lang.AssertionError:
> > > >> > Expected
> > > >> > > >> > >> 0000472128
> > > >> > > >> > >> > > but
> > > >> > > >> > >> > > > > got 0000472127
> > > >> > > >> > >> > > > >         at
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:50)
> > > >> > > >> > >> > > > >         at
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.PerformanceEvaluationCommons.assertKey(PerformanceEvaluationCommons.java:45)
> > > >> > > >> > >> > > > >         at
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.HFilePerformanceEvaluation$UniformRandomReadBenchmark.doRow(HFilePerformanceEvaluation.java:295)
> > > >> > > >> > >> > > > >         at
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.HFilePerformanceEvaluation$RowOrientedBenchmark.run(HFilePerformanceEvaluation.java:169)
> > > >> > > >> > >> > > > >         at
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.HFilePerformanceEvaluation.runBenchmark(HFilePerformanceEvaluation.java:121)
> > > >> > > >> > >> > > > >         at
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> org.apache.hadoop.hbase.HFilePerformanceEvaluation$2.run(HFilePerformanceEvaluation.java:87)
> > > >> > > >> > >> > > > >         at java.lang.Thread.run(Thread.java:744)
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > Also, on the performance side, I got very
> strange
> > > >> > results.
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > Some tests are as fast as 0.98.6, some are WAY
> > > >> slower...
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > Results are there:
> http://tiny.cloudera.com/sjeyn
> > I
> > > >> will
> > > >> > > >> > restart
> > > >> > > >> > >> the
> > > >> > > >> > >> > > > tests
> > > >> > > >> > >> > > > > tomorrow and get new numbers by mid next week.
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > Appart from that, it works. And since it's a dev
> > > >> release,
> > > >> > > >> I'm +1
> > > >> > > >> > >> with
> > > >> > > >> > >> > > it.
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > JM
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > > 2014-09-19 21:28 GMT-04:00 Andrew Purtell <
> > > >> > > >> apurtell@apache.org
> > > >> > > >> > >> > <javascript:;>>:
> > > >> > > >> > >> > > > >
> > > >> > > >> > >> > > > >> +1
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >> Checked file sums and signature
> > > >> > > >> > >> > > > >> Unpacked tarball, looked at directory layout
> > > >> > > >> > >> > > > >> Started up a minicluster in tree, did a quick
> > > ingest
> > > >> > test,
> > > >> > > >> it
> > > >> > > >> > >> > > succeeded
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >> On Tue, Sep 16, 2014 at 1:52 PM, Enis Söztutar
> <
> > > >> > > >> > enis@apache.org
> > > >> > > >> > >> > <javascript:;>>
> > > >> > > >> > >> > > wrote:
> > > >> > > >> > >> > > > >> > I am very happy to say that the second HBase
> > > 0.99.0
> > > >> > > >> release
> > > >> > > >> > >> > > candidate
> > > >> > > >> > >> > > > >> (RC1)
> > > >> > > >> > >> > > > >> > is
> > > >> > > >> > >> > > > >> > available for download at
> > > >> > > >> > >> > > > >>
> https://people.apache.org/~enis/hbase-0.99.0RC1/
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Maven artifacts are also available in the
> > > temporary
> > > >> > > >> > repository
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> >
> > > >> > > >> >
> > > >> > >
> > > >>
> > https://repository.apache.org/content/repositories/orgapachehbase-1039/
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Signed with my code signing key E964B5FF. Can
> > be
> > > >> found
> > > >> > > >> here:
> > > >> > > >> > >> > > > >> >
> > > https://people.apache.org/keys/committer/enis.asc
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Signed tag in the repository can be found
> here:
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=hbase.git;a=tag;h=336a63b5f58bb1c4859e3c2b6a2f5c0d68a7d9e1
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > NOTE IN GIGANTIC LETTERS THAT THIS IS A
> > DEVELOPER
> > > >> > > RELEASE.
> > > >> > > >> > >> > > > >> > DO NOT USE THIS RELEASE IN PRODUCTION.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > HBase 0.99.0 is a "developer preview"
> release,
> > > and
> > > >> an
> > > >> > > >> > >> odd-numbered
> > > >> > > >> > >> > > > >> release
> > > >> > > >> > >> > > > >> > as
> > > >> > > >> > >> > > > >> > defined in
> > > >> > > >> > >> > > > >>
> > > >> > > >>
> https://hbase.apache.org/book/upgrading.html#hbase.versioning.
> > > >> > > >> > >> > > > >> > This release IS NOT intended for production
> > use,
> > > and
> > > >> > > does
> > > >> > > >> not
> > > >> > > >> > >> > > contain
> > > >> > > >> > >> > > > any
> > > >> > > >> > >> > > > >> > backwards or forwards compatibility
> guarantees
> > > (even
> > > >> > > >> within
> > > >> > > >> > >> minor
> > > >> > > >> > >> > > > >> versions
> > > >> > > >> > >> > > > >> > 0.99.x). Please refrain from deploying this
> > over
> > > >> > > important
> > > >> > > >> > >> data.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > 0.99.0 release is provided from branch-1,
> which
> > > will
> > > >> > be
> > > >> > > >> the
> > > >> > > >> > >> basis
> > > >> > > >> > >> > > for
> > > >> > > >> > >> > > > >> > HBase-1.0
> > > >> > > >> > >> > > > >> > release. A few 0.99.x releases are planned
> > before
> > > >> 1.0.
> > > >> > > The
> > > >> > > >> > >> reason
> > > >> > > >> > >> > > for
> > > >> > > >> > >> > > > >> > doing a developer preview release is to get
> > more
> > > >> > testing
> > > >> > > >> for
> > > >> > > >> > >> the
> > > >> > > >> > >> > > > branch-1
> > > >> > > >> > >> > > > >> > code
> > > >> > > >> > >> > > > >> > that will be released soon as HBase-1.0.0.
> > Thus,
> > > all
> > > >> > > >> > >> contribution
> > > >> > > >> > >> > in
> > > >> > > >> > >> > > > >> terms
> > > >> > > >> > >> > > > >> > of
> > > >> > > >> > >> > > > >> > testing, benchmarking, checking API / source
> > > /wire
> > > >> > > >> > >> compatibility,
> > > >> > > >> > >> > > > >> checking
> > > >> > > >> > >> > > > >> > out
> > > >> > > >> > >> > > > >> > documentation and further code contribution
> is
> > > >> highly
> > > >> > > >> > >> appreciated.
> > > >> > > >> > >> > > 1.0
> > > >> > > >> > >> > > > >> will
> > > >> > > >> > >> > > > >> > be
> > > >> > > >> > >> > > > >> > the first series in the 1.x line of releases
> > > which
> > > >> are
> > > >> > > >> > >> expected to
> > > >> > > >> > >> > > > keep
> > > >> > > >> > >> > > > >> > compatibility with previous 1.x releases.
> Thus
> > > it is
> > > >> > > very
> > > >> > > >> > >> > important
> > > >> > > >> > >> > > to
> > > >> > > >> > >> > > > >> > check
> > > >> > > >> > >> > > > >> > the client side and server side APIs for
> > > >> compatibility
> > > >> > > and
> > > >> > > >> > >> > > > >> maintainability
> > > >> > > >> > >> > > > >> > concerns for future releases.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > 0.99.0 contains slightly more than 1K issues
> > > >> resolved
> > > >> > > with
> > > >> > > >> > many
> > > >> > > >> > >> > > > >> > improvements
> > > >> > > >> > >> > > > >> > and bug fixes. The theme of (eventual) 1.0
> > > release
> > > >> is
> > > >> > to
> > > >> > > >> > >> become a
> > > >> > > >> > >> > > > stable
> > > >> > > >> > >> > > > >> > base
> > > >> > > >> > >> > > > >> > for future 1.x series of releases. 1.0
> release
> > > will
> > > >> > aim
> > > >> > > to
> > > >> > > >> > >> achieve
> > > >> > > >> > >> > > at
> > > >> > > >> > >> > > > >> least
> > > >> > > >> > >> > > > >> > the same level of stability of 0.98 releases
> > > without
> > > >> > > >> > >> introducing
> > > >> > > >> > >> > too
> > > >> > > >> > >> > > > many
> > > >> > > >> > >> > > > >> > new
> > > >> > > >> > >> > > > >> > features.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Some work has been under way to clearly mark
> > and
> > > >> > > >> > differentiate
> > > >> > > >> > >> > > client
> > > >> > > >> > >> > > > >> > facing
> > > >> > > >> > >> > > > >> > APIs, and redefine some of the client
> > interfaces
> > > for
> > > >> > > >> > improving
> > > >> > > >> > >> > > > semantics,
> > > >> > > >> > >> > > > >> > easy
> > > >> > > >> > >> > > > >> > of use and maintainability. 0.99.0 contains
> > some
> > > of
> > > >> > the
> > > >> > > >> > >> completed
> > > >> > > >> > >> > > > items,
> > > >> > > >> > >> > > > >> > and
> > > >> > > >> > >> > > > >> > the rest can be found HBASE-10602.
> > > Marking/remarking
> > > >> > of
> > > >> > > >> > >> interfaces
> > > >> > > >> > >> > > > with
> > > >> > > >> > >> > > > >> > InterfaceAudience has also been going on,
> which
> > > will
> > > >> > > >> identify
> > > >> > > >> > >> > areas
> > > >> > > >> > >> > > > for
> > > >> > > >> > >> > > > >> > compatibility (with clients, coprocessors and
> > > >> > dependent
> > > >> > > >> > >> projects
> > > >> > > >> > >> > > like
> > > >> > > >> > >> > > > >> > Phoenix)
> > > >> > > >> > >> > > > >> > for future releases.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Starting with 0.99.0, the HBase master server
> > and
> > > >> > backup
> > > >> > > >> > master
> > > >> > > >> > >> > > > servers
> > > >> > > >> > >> > > > >> > will
> > > >> > > >> > >> > > > >> > also act as a region server. RPC port and
> info
> > > port
> > > >> > for
> > > >> > > >> web
> > > >> > > >> > UI
> > > >> > > >> > >> is
> > > >> > > >> > >> > > > shared
> > > >> > > >> > >> > > > >> > for
> > > >> > > >> > >> > > > >> > the master and region server roles. Active
> > master
> > > >> and
> > > >> > > >> backup
> > > >> > > >> > >> > masters
> > > >> > > >> > >> > > > by
> > > >> > > >> > >> > > > >> > default
> > > >> > > >> > >> > > > >> > will NOT be hosting any regions unless
> > configured
> > > >> > > >> otherwise.
> > > >> > > >> > >> See
> > > >> > > >> > >> > > > >> > HBASE-10569
> > > >> > > >> > >> > > > >> > and HBASE-11604 for more details.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > This release contains Phase 1 items for
> > > experimental
> > > >> > > "Read
> > > >> > > >> > >> > > > availability
> > > >> > > >> > >> > > > >> > using
> > > >> > > >> > >> > > > >> > timeline consistent region replicas"
> feature. A
> > > >> region
> > > >> > > >> can be
> > > >> > > >> > >> > hosted
> > > >> > > >> > >> > > > in
> > > >> > > >> > >> > > > >> > multiple region servers in read-only mode.
> One
> > of
> > > >> the
> > > >> > > >> > replicas
> > > >> > > >> > >> for
> > > >> > > >> > >> > > the
> > > >> > > >> > >> > > > >> > region
> > > >> > > >> > >> > > > >> > will be primary, accepting writes, and other
> > > >> replicas
> > > >> > > >> will be
> > > >> > > >> > >> > > sharing
> > > >> > > >> > >> > > > the
> > > >> > > >> > >> > > > >> > same
> > > >> > > >> > >> > > > >> > data files. Read requests can be done against
> > any
> > > >> > > replica
> > > >> > > >> for
> > > >> > > >> > >> the
> > > >> > > >> > >> > > > region
> > > >> > > >> > >> > > > >> > with
> > > >> > > >> > >> > > > >> > backup RPCs for high availability with
> timeline
> > > >> > > >> consistency
> > > >> > > >> > >> > > > guarantees.
> > > >> > > >> > >> > > > >> > More
> > > >> > > >> > >> > > > >> > information can be found at HBASE-10070.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Other notable improvements in this release
> are
> > > >> > > >> > >> > > > >> >  - Automatic tuning of global memstore and
> > block
> > > >> cache
> > > >> > > >> sizes
> > > >> > > >> > >> > > > >> >  - Various security, tags and visibility
> labels
> > > >> > > >> improvements
> > > >> > > >> > >> > > > >> >  - Bucket cache improvements (usability and
> > > >> compressed
> > > >> > > >> data
> > > >> > > >> > >> > blocks)
> > > >> > > >> > >> > > > >> >  - A new pluggable replication endpoint to
> plug
> > > in
> > > >> to
> > > >> > > >> HBase's
> > > >> > > >> > >> > > > >> inter-cluster
> > > >> > > >> > >> > > > >> >    replication to replicate to a custom data
> > > store
> > > >> > > >> > >> > > > >> >  - A Dockerfile to easily build and run HBase
> > > from
> > > >> > > source
> > > >> > > >> > >> > > > >> >  - Internal refactoring for abstracting away
> > > >> zookeeper
> > > >> > > >> usage
> > > >> > > >> > >> > > > >> >  - Truncate table command
> > > >> > > >> > >> > > > >> >  - Region assignment to use hbase:meta table
> > > instead
> > > >> > of
> > > >> > > >> > >> zookeeper
> > > >> > > >> > >> > > for
> > > >> > > >> > >> > > > >> faster
> > > >> > > >> > >> > > > >> >    region assignment (disabled by default)
> > > >> > > >> > >> > > > >> >  - Better support for Cell interface
> internally
> > > in
> > > >> > read
> > > >> > > >> and
> > > >> > > >> > >> write
> > > >> > > >> > >> > > > paths
> > > >> > > >> > >> > > > >> for
> > > >> > > >> > >> > > > >> >    better performance and flexibility
> > > >> > > >> > >> > > > >> >  - Combining internal mvcc and seqId
> > > infrastructure
> > > >> > > >> > >> > > > >> >  - Extensive documentation improvements
> > > >> > > >> > >> > > > >> >  - Numerous improvements in other areas and
> bug
> > > >> fixes.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > The release has these changes in default
> > > behaviour:
> > > >> > > >> > >> > > > >> >  - hfile version 3 has been enabled by
> default
> > > >> > > >> > >> > > > >> >  - Distributed log replay has been enabled by
> > > >> default
> > > >> > > >> > >> > > > >> >  - Slab cache has been removed (use bucket
> > cache
> > > >> > > instead)
> > > >> > > >> > >> > > > >> >  - Some client facing APIs (HTableInterface,
> > etc)
> > > >> has
> > > >> > > been
> > > >> > > >> > >> > > deprecated
> > > >> > > >> > >> > > > and
> > > >> > > >> > >> > > > >> >    replaced.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > We have also adopted an orca as the mascot
> for
> > > >> Apache
> > > >> > > >> HBase.
> > > >> > > >> > >> See
> > > >> > > >> > >> > it
> > > >> > > >> > >> > > at
> > > >> > > >> > >> > > > >> > https://hbase.apache.org/
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Thanks for everybody who have contributed to
> > this
> > > >> > > release.
> > > >> > > >> > Full
> > > >> > > >> > >> > list
> > > >> > > >> > >> > > > of
> > > >> > > >> > >> > > > >> the
> > > >> > > >> > >> > > > >> > issues
> > > >> > > >> > >> > > > >> > can be found here:
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> >
> > > >> > > >>
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325675
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Compatibility
> > > >> > > >> > >> > > > >> > -------------
> > > >> > > >> > >> > > > >> > 0.99.x series of releases are preview of
> > upcoming
> > > >> 1.0
> > > >> > > >> > release.
> > > >> > > >> > >> > > > HBase-1.0
> > > >> > > >> > >> > > > >> is
> > > >> > > >> > >> > > > >> > planned to be wire compatible with 0.98.x and
> > > 0.96.x
> > > >> > > >> > releases.
> > > >> > > >> > >> > > Clients
> > > >> > > >> > >> > > > >> and
> > > >> > > >> > >> > > > >> > servers running in different versions as long
> > as
> > > new
> > > >> > > >> features
> > > >> > > >> > >> are
> > > >> > > >> > >> > > not
> > > >> > > >> > >> > > > >> used
> > > >> > > >> > >> > > > >> > should be possible.
> > > >> > > >> > >> > > > >> > A rolling upgrade from 0.98.x clusters to
> > 0.99.0
> > > is
> > > >> > > >> supported
> > > >> > > >> > >> as
> > > >> > > >> > >> > > well.
> > > >> > > >> > >> > > > >> > 0.99.0
> > > >> > > >> > >> > > > >> > introduces a new file format (hfile v3) that
> is
> > > >> > enabled
> > > >> > > by
> > > >> > > >> > >> default
> > > >> > > >> > >> > > > that
> > > >> > > >> > >> > > > >> > 0.96.x code cannot read. Rolling upgrade from
> > > 0.96
> > > >> > > >> directly
> > > >> > > >> > to
> > > >> > > >> > >> > 0.99
> > > >> > > >> > >> > > is
> > > >> > > >> > >> > > > >> not
> > > >> > > >> > >> > > > >> > tested.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Direct migration from 0.94.x is also
> supported
> > > but
> > > >> it
> > > >> > IS
> > > >> > > >> NOT
> > > >> > > >> > >> > tested.
> > > >> > > >> > >> > > > >> Before
> > > >> > > >> > >> > > > >> > HBase-1.0 we expect to complete the test.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Binary compatibility at the Java API layer
> with
> > > >> > earlier
> > > >> > > >> > >> versions
> > > >> > > >> > >> > > > (0.98.x,
> > > >> > > >> > >> > > > >> > 0.96.x and 0.94.x) is not supported. You may
> > > have to
> > > >> > > >> > recompile
> > > >> > > >> > >> > your
> > > >> > > >> > >> > > > >> client
> > > >> > > >> > >> > > > >> > code and any server side code (coprocessors,
> > > filters
> > > >> > > etc)
> > > >> > > >> > >> > referring
> > > >> > > >> > >> > > to
> > > >> > > >> > >> > > > >> > hbase jars.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > 0.99.0 release introduces new APIs, and
> > > deprecates
> > > >> > some
> > > >> > > of
> > > >> > > >> > >> > > > commonly-used
> > > >> > > >> > >> > > > >> > client side APIs (HTableInterface, etc).
> > However,
> > > >> > client
> > > >> > > >> side
> > > >> > > >> > >> code
> > > >> > > >> > >> > > is
> > > >> > > >> > >> > > > >> > expected
> > > >> > > >> > >> > > > >> > to be source compatible with earlier
> versions.
> > > >> However
> > > >> > > we
> > > >> > > >> > >> advise
> > > >> > > >> > >> > to
> > > >> > > >> > >> > > > >> update
> > > >> > > >> > >> > > > >> > your application to use the new style of
> APIs,
> > > since
> > > >> > > >> > deprecated
> > > >> > > >> > >> > APIs
> > > >> > > >> > >> > > > >> might
> > > >> > > >> > >> > > > >> > be removed in future releases (2.x).
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Supported Hadoop versions
> > > >> > > >> > >> > > > >> > -------------------------
> > > >> > > >> > >> > > > >> > 0.99.0 release drops support for Hadoop-1.x
> > > >> releases.
> > > >> > > Only
> > > >> > > >> > >> > > Hadoop-2.x
> > > >> > > >> > >> > > > >> > releases are supported. Hadoop-2.4.x and
> > > >> Hadoop-2.5.x
> > > >> > > >> > releases
> > > >> > > >> > >> are
> > > >> > > >> > >> > > the
> > > >> > > >> > >> > > > >> > most tested hadoop releases and we recommend
> > > running
> > > >> > > with
> > > >> > > >> > those
> > > >> > > >> > >> > > > versions.
> > > >> > > >> > >> > > > >> > Earlier Hadoop-2 based releases (hadoop-2.2.x
> > and
> > > >> > 2.3.x)
> > > >> > > >> are
> > > >> > > >> > >> not
> > > >> > > >> > >> > > > tested
> > > >> > > >> > >> > > > >> to
> > > >> > > >> > >> > > > >> > the full
> > > >> > > >> > >> > > > >> > extend. More information can be found here:
> > > >> > > >> > >> > > > >> >
> > > >> > https://hbase.apache.org/book/configuration.html#hadoop
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Supported Java versions
> > > >> > > >> > >> > > > >> > -------------------------
> > > >> > > >> > >> > > > >> > 0.99.0 release drops support for JDK6. Only
> > JDK7
> > > is
> > > >> > > >> > supported.
> > > >> > > >> > >> > JDK8
> > > >> > > >> > >> > > > >> support
> > > >> > > >> > >> > > > >> > is experimental. More information can be
> found
> > > here:
> > > >> > > >> > >> > > > >> >
> > > >> https://hbase.apache.org/book/configuration.html#java
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Voting
> > > >> > > >> > >> > > > >> > ------
> > > >> > > >> > >> > > > >> > Please try to test and vote on this release
> by
> > > >> > > 09/19/2014
> > > >> > > >> > >> 11:59PM
> > > >> > > >> > >> > > PDT.
> > > >> > > >> > >> > > > >> > This is a developer (beta) release, so we
> have
> > a
> > > >> > shorter
> > > >> > > >> vote
> > > >> > > >> > >> > time.
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > [] +1 Release the artifacts as 0.99.0
> > > >> > > >> > >> > > > >> > [] -1 DO NOT release the artifacts as 0.99.0,
> > > >> > because...
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Thanks for helping to get a stable release
> out!
> > > >> > > >> > >> > > > >> >
> > > >> > > >> > >> > > > >> > Enis
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >> --
> > > >> > > >> > >> > > > >> Best regards,
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >>    - Andy
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > > >> Problems worthy of attack prove their worth by
> > > hitting
> > > >> > > >> back. -
> > > >> > > >> > >> Piet
> > > >> > > >> > >> > > > >> Hein (via Tom White)
> > > >> > > >> > >> > > > >>
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > --
> > > >> > > >> > >> > > > Best regards,
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > >    - Andy
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > > > Problems worthy of attack prove their worth by
> > hitting
> > > >> > back.
> > > >> > > -
> > > >> > > >> > Piet
> > > >> > > >> > >> > > > Hein (via Tom White)
> > > >> > > >> > >> > > >
> > > >> > > >> > >> > >
> > > >> > > >> > >> >
> > > >> > > >> > >>
> > > >> > > >> > >
> > > >> > > >> > >
> > > >> > > >> >
> > > >> > > >>
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> > >
> > >
> > > --
> > > Best regards,
> > >
> > >    - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> > > Hein (via Tom White)
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message