hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Re: [VOTE] The 1st hbase 0.94.15 release candidate is available for download
Date Thu, 19 Dec 2013 16:53:59 GMT
Thanks JM.

You did a "raw" scan below. It'll return to you exactly what is there, so you'll see the 3
versions before you compact, that is by design.
java.lang.AbstractMethodError looks like an issue local to your install. I'll check.

IntegrationTestLoadAndVerify is interesting. Did that pass reliably in older releases of 0.94
(0.94.14 or 0.94.13)?

-- Lars

 From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
To: dev <dev@hbase.apache.org>; lars hofhansl <larsh@apache.org> 
Sent: Thursday, December 19, 2013 7:01 AM
Subject: Re: [VOTE] The 1st hbase 0.94.15 release candidate is available for download

tl;tr see arrow below.

Downloaded and checked signature for bother vanilla and secured. Passed.
Random checked documentation and CHANGES.txt. Passed

On a single node cluster:
Ran the tests. All passed.
Ran IntegrationTestLoadAndVerify. Got  REFERENCES_CHECKED=9855424, expected 10000000? Failed?
Ran IntegrationTestBigLinkedList. Passed.
Ran HBCK after those tests and got many errors about _original-evil-name and clone tables.
Cleared everything, restarted HBase. Re-ran IntegrationTestBigLinkedList, HBCK ok. Re-ran
IntegrationTestLoadAndVerify, failed again:
13/12/18 21:24:24 ERROR test.IntegrationTestBigLinkedList$Verify: Expected referenced count
does not match with actual referenced count. expected referenced=3000000 ,actual=9000000
Exception in thread "main" java.lang.RuntimeException: Verify.verify failed
    at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.runVerify(IntegrationTestBigLinkedList.java:724)
    at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.run(IntegrationTestBigLinkedList.java:757)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList.run(IntegrationTestBigLinkedList.java:1069)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList.main(IntegrationTestBigLinkedList.java:1073)

But now HBCK is clean. Figured that HBCK issue is because of some leftover from org.apache.hadoop.hbase.regionserver.TestStoreFile
who is writting in the same directory as the default standalone HBase.

From the shell, create a table 15 regions, put, compact, scan, etc. Table definition is VERSIONS
=> 2. However, scan 't1', {RAW => true, VERSIONS => 10} still return 3 versions even
after flush/compact/major_compact:
hbase(main):034:0> scan 't1', {RAW => true, VERSIONS => 10}

column=f1:c1, timestamp=1387421969489, value=value                                                                                                                     

column=f1:c1, timestamp=1387421969337, value=value                                                                                                                     

column=f1:c1, timestamp=1387421969162, value=value                                                                                                                     

1 row(s) in 0.0570 seconds

Will have expected only 2 to be return.

Stopped HBase, checked the log, everything is fine.

Now on a 7 nodes cluster:

Deployed jars and did rolling restart on a 0.94.14 cluster. Passed.

Configured default balancer, merged a 60 region table to a single region, restarted the cluster,
all fine.

major_compact the table to get it split into 60 regions, balancer, all fine except that balancer
need to be run twice to get correct balancing.

Some "No serialized HRegionInfo in keyvalues" in the logs not related to the tables I'm "playing"

Restored customized balancer, restarted, rebalanced, all fine.
Ran IntegrationTestLoadAndVerify. Got  REFERENCES_CHECKED=9855645, expected 10000000? Failed?

Ran IntegrationTestBigLinkedList. Passed.

Last, I tried to run IntegrationTestsDriver but it failed. I need to look at that.

hbase@node3:~/hbase-0.94.3$ bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver
Exception in thread "main" java.lang.AbstractMethodError: org.apache.hadoop.hbase.util.AbstractHBaseTool.doWork()V
    at org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:103)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.hbase.IntegrationTestsDriver.main(IntegrationTestsDriver.java:47)

=====> tl;tr:

- Small issue with balancer when 60 regions assigned to a single server. Need to run twice
to get that correctly balanced;

- Leftover in the wrong place from  org.apache.hadoop.hbase.regionserver.TestStoreFile;
- Table with VERSIONS => 2 returns 3 versions instead of 2;
- IntegrationTestsDriver not running.

I don't think there is anything here to stop the release but there is still few things that
need to be looked at.


2013/12/18 lars hofhansl <larsh@apache.org>

The 1st 0.94.15 RC is available for download at http://people.apache.org/~larsh/hbase-0.94.15-rc0/
>Signed with my code signing key: C7CFE328
>HBase 0.94.15 is a bug fix release along with some performance improvements:
>    [HBASE-7886] - [replication] hlog zk node will not be deleted if client roll hlog
>    [HBASE-9485] - TableOutputCommitter should implement recovery if we don't want
jobs to start from 0 on RM restart
>    [HBASE-9995] - Not stopping ReplicationSink when using custom implementation for
the ReplicationSink
>    [HBASE-10014] - HRegion#doMiniBatchMutation rollbacks the memstore even if there
is nothing to rollback.
>    [HBASE-10015] - Replace intrinsic locking with explicit locks in StoreScanner
>    [HBASE-10026] - HBaseAdmin#createTable could fail if region splits too fast
>    [HBASE-10046] - Unmonitored HBase service could accumulate Status objects and OOM
>    [HBASE-10057] - TestRestoreFlushSnapshotFromClient and TestRestoreSnapshotFromClient
fail to finish occasionally
>    [HBASE-10061] - TableMapReduceUtil.findOrCreateJar calls updateMap(null, ) resulting
in thrown NPE
>    [HBASE-10064] - AggregateClient.validateParameters can throw NPE
>    [HBASE-10089] - Metrics intern table names cause eventual permgen OOM in 0.94
>    [HBASE-10111] - Verify that a snapshot is not corrupted before restoring it
>    [HBASE-10112] - Hbase rest query params for maxVersions and maxValues are not parsed
>    [HBASE-10117] - Avoid synchronization in HRegionScannerImpl.isFilterDone
>    [HBASE-10120] - start-hbase.sh doesn't respect --config in non-distributed mode
>    [HBASE-10179] - HRegionServer underreports readRequestCounts by 1 under certain
>    [HBASE-10181] - HBaseObjectWritable.readObject catches DoNotRetryIOException and
wraps it back in a regular IOException
>    [HBASE-9931] - Optional setBatch for CopyTable to copy large rows in batches
>    [HBASE-10001] - Add a coprocessor to help testing the performances without taking
into account the i/o
>    [HBASE-10007] - PerformanceEvaluation: Add sampling and latency collection to randomRead
>    [HBASE-10010] - eliminate the put latency spike on the new log file beginning
>    [HBASE-10048] - Add hlog number metric in regionserver
>    [HBASE-10049] - Small improvments in region_mover.rb
>    [HBASE-10093] - Unregister ReplicationSource metric bean when the replication source
thread is terminated
>    [HBASE-9047] - Tool to handle finishing replication when the cluster is offline
>    [HBASE-10119] - Allow HBase coprocessors to clean up when they fail
>    [HBASE-9927] - ReplicationLogCleaner#stop() calls HConnectionManager#deleteConnection()
>    [HBASE-9986] - Incorporate HTTPS support for HBase (0.94 port)
>    [HBASE-10058] - Test for HBASE-9915 (avoid reading index blocks)
>    [HBASE-10189] - Intermittent TestReplicationSyncUpTool failure
>The list of changes is also available here: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325559
>Here're the jenkins runs for this RC: https://builds.apache.org/job/HBase-0.94.15/2/ and
>Please try out the RC, check out the doc, take it for a spin, etc, and vote +1/-1 by EOD
December 27th on whether we should release this as 0.94.15. (9 days because of the holidays)
>-- Lars
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message