hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Re: [VOTE] The 1st hbase 0.94.19 release candidate is available for download
Date Fri, 25 Apr 2014 02:55:48 GMT
Wow. Thanks JM!

-- Lars


________________________________
From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
To: dev <dev@hbase.apache.org> 
Sent: Thursday, April 24, 2014 10:09 AM
Subject: Re: [VOTE] The 1st hbase 0.94.19 release candidate is available for download


So. I'm done with my heavy duty release test for 0.94.19.

tl.tr: +1 ;)

Here are the details.
Downloaded the jar, checked the signature, the CHANGES.txt file, the
documentation (random pickup) -> Passed.
Run the test suite -> Passed.
Tests run: 1550, Failures: 0, Errors: 0, Skipped: 16
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 1:16:23.687s
[INFO] Finished at: Wed Apr 23 12:02:26 EDT 2014
[INFO] Final Memory: 29M/983M
[INFO]
------------------------------------------------------------------------

Run PE over few days and compared with 0.94.18.
Got one exception on GaussianRandomReadBenchmark for
HFilePerformanceEvaluation. This occurs multiple times but was always the
same exception:
org.apache.hadoop.hbase.io.hfile.AbstractHFileReader$NotSeekedException:
Not seeked to a key/value
        at
org.apache.hadoop.hbase.io.hfile.AbstractHFileReader$Scanner.assertSeeked(AbstractHFileReader.java:320)
        at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.getKey(HFileReaderV2.java:650)
        at
org.apache.hadoop.hbase.HFilePerformanceEvaluation$GaussianRandomReadBenchmark.doRow(HFilePerformanceEvaluation.java:350)
        at
org.apache.hadoop.hbase.HFilePerformanceEvaluation$RowOrientedBenchmark.run(HFilePerformanceEvaluation.java:169)
        at
org.apache.hadoop.hbase.HFilePerformanceEvaluation.runBenchmark(HFilePerformanceEvaluation.java:121)
        at
org.apache.hadoop.hbase.HFilePerformanceEvaluation$3.run(HFilePerformanceEvaluation.java:97)
        at java.lang.Thread.run(Thread.java:744)

Performance wise this release is very similar to the previous one (>0% mean
0.94.19 is faster):

0.94.180.94.19
FilteredScanTest0,23 0,230,01%RandomReadTest 8258250,01% RandomSeekScanTest
173178 2,89%RandomScanWithRange10Test282 2861,58%RandomScanWithRange100Test149
147-1,10% RandomScanWithRange1000Test37,5739,40
4,85%SequentialReadTest1 2061 225
1,51%SequentialWriteTest 13 68713 8261,02% RandomWriteTest14 09213 574-3,68%
GaussianRandomReadBenchmark9 400 9 395-0,05%SequentialReadBenchmark3 035 361
3 009 210-0,86% SequentialWriteBenchmark909 881909 579 -0,03%
UniformRandomReadBenchmark10 312 10 3540,41%UniformRandomSmallScan 233 141
233 9310,34% LoadTestToolreal 19m24.218s
user 36m17.208s
sys 11m45.128sreal 19m11.070s
user 37m5.328s
sys 11m10.724s
IntegrationTestLoadAnVerify real 4m7.922s
user 1m31.100s
sys 0m7.324sreal 4m7.909s
user 1m30.668s
sys 0m7.136s
HLogPerformanceEvaluation10571,8310597,149
0,24%IntegrationTestBigLinkedListreal
6m20.893s
user 2m55.068s
sys 0m10.728s real 6m26.026s
user 3m0.156s
sys 0m10.436s

Ran LoadTestTool, IntegrationTestLoadAndVerify, HLogPerformanceEvaluation,
IntegrationTestBigLinkedList. I consider them as passed since I have the
same results as for 0.94.18 but I still have the
IntegrationTestLoadAndVerify reporting wrong results.

14/04/23 22:45:03 INFO mapred.JobClient:
org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify$Counters
14/04/23 22:45:03 INFO mapred.JobClient:     ROWS_WRITTEN=0
14/04/23 22:45:03 INFO mapred.JobClient:     REFERENCES_CHECKED=9855522
(Should be 10000000)

Tried to manually create table, put, scan, flush, get, compact, drop, etc.
All passed.

Deployed on 8 node cluster. RSs rolling restart went well.
Stopped the cluster, merged a 32 region table to a single region, restart,
major_compact, got it splitted correctly.

Tried the default load balancer but did not worked well the first time.
Initial state:
node7.google.com <http://node7.distparser.com>,60020,1398349950181     32

After first run:
Regions by Region Server
Region Server    Region Count
node1. <http://node1.distparser.com>google
<http://node7.distparser.com>.com<http://node1.distparser.com>,60020,1398349948960
    2
node2. <http://node2.distparser.com>google
<http://node7.distparser.com>.com<http://node2.distparser.com>,60020,1398349952018
    3
node3. <http://node3.distparser.com>google
<http://node7.distparser.com>.com<http://node3.distparser.com>,60020,1398349947506
    2
node4. <http://node4.distparser.com>google
<http://node7.distparser.com>.com<http://node4.distparser.com>,60020,1398349949141
    3
node5. <http://node5.distparser.com>google
<http://node7.distparser.com>.com<http://node5.distparser.com>,60020,1398349946262
    2
node6. <http://node6.distparser.com>google
<http://node7.distparser.com>.com<http://node6.distparser.com>,60020,1398349951095
    2
node7. <http://node7.distparser.com>google
<http://node7.distparser.com>.com<http://node7.distparser.com>,60020,1398349950181
    16
node8. <http://node8.distparser.com>google
<http://node7.distparser.com>.com<http://node8.distparser.com>,60020,1398349948480
    2

After 2nd run:
Regions by Region Server
Region Server    Region Count
node1. <http://node1.distparser.com>google
<http://node7.distparser.com>.com<http://node1.distparser.com>,60020,1398349948960
    4
node2. <http://node2.distparser.com>google
<http://node7.distparser.com>.com<http://node2.distparser.com>,60020,1398349952018
    4
node3. <http://node3.distparser.com>google
<http://node7.distparser.com>.com<http://node3.distparser.com>,60020,1398349947506
    4
node4. <http://node4.distparser.com>google
<http://node7.distparser.com>.com<http://node4.distparser.com>,60020,1398349949141
    4
node5. <http://node5.distparser.com>google
<http://node7.distparser.com>.com<http://node5.distparser.com>,60020,1398349946262
    4
node6. <http://node6.distparser.com>google
<http://node7.distparser.com>.com<http://node6.distparser.com>,60020,1398349951095
    4
node7. <http://node7.distparser.com>google
<http://node7.distparser.com>.com<http://node7.distparser.com>,60020,1398349950181
    4
node8. <http://node8.distparser.com>google
<http://node7.distparser.com>.com<http://node8.distparser.com>,60020,1398349948480
    4

But that's not a show stopper.
Restored my default balancer, restart the cluster, got everything balanced
correctly. Might be nice to have a way to change the balancer without
having to restart HBase...

I checked the webUI, the logs and HBCK all over the process and they are
all reporting well.

Last, integration test is running for the 2 last hours with no issues.

So I'm +1 with this release.

JM



2014-04-24 2:40 GMT-04:00 Srikanth Srungarapu <srikanth235@gmail.com>:

> +1 (non-binding)
>
> - Verified the signature and md5
>
> - Ran test suite on both local and distributed mode (all got passed with
> two being skipped)
>
> - Inspected UI and CHANGES.txt.
>
> Thanks,
> Srikanth.
>
>
> On Wed, Apr 23, 2014 at 5:54 PM, lars hofhansl <larsh@apache.org> wrote:
>
> > Thanks Ted and Esteban!
> >
> >
> > -- Lars
> >
> >
> >
> > ________________________________
> >  From: Esteban Gutierrez <esteban@cloudera.com>
> > To: "dev@hbase.apache.org" <dev@hbase.apache.org>
> > Cc: lars hofhansl <larsh@apache.org>
> > Sent: Tuesday, April 22, 2014 9:36 PM
> > Subject: Re: [VOTE] The 1st hbase 0.94.19 release candidate is available
> > for download
> >
> >
> > +1 (non-binding)
> >
> > signature good, all tests passed on first run (2 skipped), ran PE
> > with SecureRpcEngine on pseudo distributed mode.
> >
> > esteban.
> >
> >
> >
> > --
> > Cloudera, Inc.
> >
> >
> >
> >
> > On Tue, Apr 22, 2014 at 3:48 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >
> > > +1
> > >
> > > - checked documentation and tarball
> > >
> > > - Ran unit test suite which passed (TestTableSnapshotInputFormatScan
> > passed
> > > on second run)
> > >
> > > - Ran in local and distributed mode
> > >
> > > Cheers
> > >
> > >
> > > On Mon, Apr 21, 2014 at 7:49 PM, lars hofhansl <larsh@apache.org>
> wrote:
> > >
> > > > The 1st 0.94.19 RC is available for download at
> > > > http://people.apache.org/~larsh/hbase-0.94.19-rc0/
> > > > Signed with my code signing key: C7CFE328
> > > >
> > > > HBase 0.94.19 is a bug fix release with 29 bug and test fixes:
> > > > Bug
> > > >     [HBASE-10118] - Major compact keeps deletes with future
> timestamps
> > > >     [HBASE-10312] - Flooding the cluster with administrative actions
> > > leads
> > > > to collapse
> > > >     [HBASE-10533] - commands.rb is giving wrong error messages on
> > > > exceptions
> > > >     [HBASE-10766] - SnapshotCleaner allows to delete referenced files
> > > >     [HBASE-10805] - Speed up KeyValueHeap.next() a bit
> > > >     [HBASE-10807] - -ROOT- still stale in table.jsp if it moved
> > > >     [HBASE-10845] - Memstore snapshot size isn't updated in
> > > > DefaultMemStore#rollback()
> > > >     [HBASE-10847] - 0.94: drop non-secure builds, make security the
> > > default
> > > >     [HBASE-10848] - Filter SingleColumnValueFilter combined with
> > > > NullComparator does not work
> > > >     [HBASE-10966] - RowCounter misinterprets column names that have
> > > colons
> > > > in their qualifier
> > > >     [HBASE-10991] - Port HBASE-10639 'Unload script displays wrong
> > counts
> > > > (off by one) when unloading regions' to 0.94
> > > >     [HBASE-11003] - ExportSnapshot is using the wrong fs when staging
> > dir
> > > > is not in fs.defaultFS
> > > >     [HBASE-11030] - HBaseTestingUtility.getMiniHBaseCluster should
be
> > > able
> > > > to return null
> > > >     [HBASE-10921] - Port HBASE-10323 'Auto detect data block encoding
> > in
> > > > HFileOutputFormat' to 0.94 / 0.96
> > > >
> > > > Test
> > > >     [HBASE-10782] - Hadoop2 MR tests fail occasionally because of
> > > > mapreduce.jobhistory.address is no set in job conf
> > > >     [HBASE-10969] - TestDistributedLogSplitting fails frequently in
> > 0.94.
> > > >     [HBASE-10982] -
> > > > TestZKProcedure.testMultiCohortWithMemberTimeoutDuringPrepare fails
> > > > frequently in 0.94
> > > >     [HBASE-10987] - Increase timeout in
> > > > TestZKLeaderManager.testLeaderSelection
> > > >     [HBASE-10988] - Properly wait for server in
> TestThriftServerCmdLine
> > > >     [HBASE-10989] - TestAccessController needs better timeout
> > > >     [HBASE-10996] - TestTableSnapshotInputFormatScan fails frequently
> > on
> > > > 0.94
> > > >     [HBASE-11010] - TestChangingEncoding is unnecessarily slow
> > > >     [HBASE-11017] - TestHRegionBusyWait.testWritesWhileScanning fails
> > > > frequently in 0.94
> > > >     [HBASE-11022] - Increase timeout for
> > > > TestHBaseFsck.testSplitDaughtersNotInMeta
> > > >     [HBASE-11024] - TestSecureLoadIncrementalHFilesSplitRecovery
> should
> > > > wait longer for ACL table
> > > >     [HBASE-11029] - Increase wait in
> > TestSplitTransactionOnCluster.split
> > > >     [HBASE-11037] - Race condition in TestZKBasedOpenCloseRegion
> > > >     [HBASE-11040] - TestAccessController, TestAccessControllerFilter,
> > and
> > > > TestTablePermissions need to wait longer to ACL table
> > > >     [HBASE-11042] - TestForceCacheImportantBlocks OOMs occasionally
> in
> > > 0.94
> > > >
> > > > Notable is HBASE-10847, which drops non-secure builds and make
> security
> > > > the default. From here on there is only one release build of HBase
> > 0.94.
> > > >
> > > > The list of changes is also available here:
> > > >
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12326287
> > > >
> > > > Here're the jenkins runs for this RC:
> > > > https://builds.apache.org/job/HBase-0.94.19/18/
> > > >
> > > > Please try out the RC, check out the doc, take it for a spin, etc,
> and
> > > > vote +1/-1 by EOD April 27th on whether we should release this as
> > > 0.94.19.
> > > >
> > > > Thanks.
> > > >
> > > > -- Lars
> > > >
> > >
> >
>

Mime
View raw message