hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: [VOTE] The 2nd hbase 0.94.17 release candidate is available for download
Date Sun, 23 Feb 2014 12:57:55 GMT
Oh, I forgot this. When running HBCK we area bit spam by this warning:
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.
14/02/22 20:31:26 INFO util.HBaseFsck: Handling overlap merges in parallel.
set hbasefsck.overlap.merge.parallel to false to run serially.

Seems it's displayed once per table. I think we can display it only once
only and then avoid the other warnings for the same.

JM


2014-02-23 7:47 GMT-05:00 Jean-Marc Spaggiari <jean-marc@spaggiari.org>:

> Tested, tl.tr: +1 ;)
>
> All tests are done with Oracle JDK 1.7.0_45 + Hadoop 1.2.1 (Except for the
> 0.96.1 test where Hadoop 2.2.0 is used)
>
> Checked the jars, the signature, the CHANGES.TXT file, the documentation
> => Passed.
> Checked logs, UI, HBCK, test suite => Passed
> Run all Performances tests => Passed. 0.94.17 performances just a bit
> under 0.94.16. RandomReadTest 15% slower with 0.94.17. Will retest it. Will
> post all the numbers in detail in my blog later today.
>
> Only concern is again the IntegrationTestLoadAndVerify result where I
> don't have the exptected result, but it's the same thing with 0.94.16 and
> 0.96.1 too. With difference Hadoop version.
>
>
> 0.94.16 + Hadoop 1.2.1:
> 14/02/21 21:22:19 INFO mapred.JobClient:
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify$Counters
> 14/02/21 21:22:19 INFO mapred.JobClient:     ROWS_WRITTEN=0
> 14/02/21 21:22:19 INFO mapred.JobClient:     REFERENCES_CHECKED=9855584
>
> 0.94.17 + Hadoop 1.2.1:
> 14/02/21 21:31:28 INFO mapred.JobClient:
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify$Counters
> 14/02/21 21:31:28 INFO mapred.JobClient:     ROWS_WRITTEN=0
> 14/02/21 21:31:28 INFO mapred.JobClient:     REFERENCES_CHECKED=9855909
>
> 0.96.1 + Hadoop 2.2.0:
> org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify$Counters
>  REFERENCES_CHECKED=9855588
>
> Create table, put, scan, compact => Passed.
> Offline merged all regions into a single one, hbck, split => Passed.
> Rolling restart => Failed once because of jar push issue (my bad), retried
> => Passed.
> Ran all tests (LoadTestTool, IntegrationTestLoadAndVerify,
> HLogPerformanceEvaluation, IntegrationTestBigLinkedList) on a 4 nodes
> cluster => Passed.
> When we click split in the UA, seems to trigger major compaction, which
> was not the case before (Not sure in which version this has been introduced)
>
> Summary:
> - IntgrationTestLoadAndVerify results still "strange";
> - Split behaviour in the UA seem the have changed.
> - +1
>
> JM
> PS: Go Canada Go!
>
> 2014-02-21 10:01 GMT-05:00 Ted Yu <yuzhihong@gmail.com>:
>
>> +1 based on the following:
>>
>> 1. Ran test suite using JDK 1.7 - pass
>> 2. compiled Phoenix with locally built 0.94.17 - pass
>> 3. checked the documentation
>>
>> Cheers
>>
>>
>> On Thu, Feb 20, 2014 at 11:18 PM, lars hofhansl <larsh@apache.org> wrote:
>>
>> > +1
>> >
>> > - checked documentation and tarball
>> >
>> > - Ran in local and distributes mode
>> > - checked the UI pages
>> >
>> > - loaded 100m KeyValues
>> > - tested scan performance
>> >
>> > - converted between FAST_DIFF and NONE encoding
>> > - tested flushes and compactions
>> >
>> >
>> > -- Lars
>> >
>> >
>> > ________________________________
>> >  From: lars hofhansl <larsh@apache.org>
>> > To: hbase-dev <dev@hbase.apache.org>
>> > Sent: Tuesday, February 18, 2014 8:31 PM
>> > Subject: [VOTE] The 2nd hbase 0.94.17 release candidate is available for
>> > download
>> >
>> >
>> > The 2nd 0.94.17 RC is available for download at
>> > http://people.apache.org/~larsh/hbase-0.94.17-rc1/
>> > Signed with my code signing key: C7CFE328
>> >
>> > HBase 0.94.17 is a bug fix release with 35 fixes:
>> >     [HBASE-7963] - HBase VerifyReplication not working when security
>> > enabled
>> >     [HBASE-10249] - TestReplicationSyncUpTool fails because failover
>> takes
>> > too long
>> >     [HBASE-10274] - MiniZookeeperCluster should close ZKDatabase when
>> > shutdown ZooKeeperServers
>> >     [HBASE-10319] - HLog should roll periodically to allow DN
>> decommission
>> > to eventually complete.
>> >     [HBASE-10320] - Avoid ArrayList.iterator() ExplicitColumnTracker
>> >     [HBASE-10335] - AuthFailedException in zookeeper may block
>> replication
>> > forever
>> >     [HBASE-10340] - [BACKPORT] HBASE-9892 Add info port to ServerName to
>> > support multi instances in a node
>> >     [HBASE-10363] - [0.94] TestInputSampler and TestInputSamplerTool
>> fail
>> > under hadoop 2.0/23 profiles.
>> >     [HBASE-10371] - Compaction creates empty hfile, then selects this
>> file
>> > for compaction and creates empty hfile and over again
>> >     [HBASE-10383] - Secure Bulk Load for 'completebulkload' fails for
>> > version 0.94.15
>> >     [HBASE-10400] - [hbck] Continue if region dir missing on region
>> merge
>> > attempt
>> >     [HBASE-10401] - [hbck] perform overlap group merges in parallel
>> >     [HBASE-10448] - ZKUtil create and watch methods don't set watch in
>> > some cases
>> >     [HBASE-10470] - Import generates huge log file while importing large
>> > amounts of data
>> >     [HBASE-10481] - API Compatibility JDiff script does not properly
>> > handle arguments in reverse order
>> >     [HBASE-10482] - ReplicationSyncUp doesn't clean up its ZK, needed
>> for
>> > tests
>> >     [HBASE-10485] - PrefixFilter#filterKeyValue() should perform
>> filtering
>> > on row key
>> >     [HBASE-10489] - TestImportExport fails in 0.94 with Hadoop2
>> >     [HBASE-10493] - InclusiveStopFilter#filterKeyValue() should perform
>> > filtering on row key
>> >     [HBASE-10501] - Improve IncreasingToUpperBoundRegionSplitPolicy to
>> > avoid too many regions
>> >     [HBASE-10505] - Import.filterKv does not call Filter.filterRowKey
>> >     [HBASE-10506] - Fail-fast if client connection is lost before the
>> real
>> > call be executed in RPC layer
>> >     [HBASE-10508] - Backport HBASE-10365 'HBaseFsck should clean up
>> > connection properly when repair is completed' to 0.94 and 0.96
>> >     [HBASE-10539] - HRegion.addAndGetGlobalMemstoreSize returns previous
>> > size
>> >     [HBASE-10545] - RS Hangs waiting on region to close on shutdown; has
>> > to timeout before can go down
>> >     [HBASE-10546] - Two scanner objects are open for each hbase map task
>> > but only one scanner object is closed
>> >     [HBASE-10551] - Change local mode back to one RS thread by default
>> >     [HBASE-10552] -
>> HFilePerformanceEvaluation.GaussianRandomReadBenchmark
>> > fails sometimes.
>> >     [HBASE-10555] - Backport HBASE-8519 to 0.94, Backup master will
>> never
>> > come up if primary master dies during initialization
>> >     [HBASE-10562] - Fix TestMultiTableInputFormat for Hadoop 2 in 0.94
>> >     [HBASE-10212] - New rpc metric: number of active handler
>> >     [HBASE-10423] - Report back the message of split or rollback failure
>> > to the master
>> >     [HBASE-10457] - Print corrupted file information in SnapshotInfo
>> tool
>> > without -file option
>> >     [HBASE-10473] - Add utility for adorning http Context
>> >     [HBASE-10480] - TestLogRollPeriod#testWithEdits may fail due to
>> > insufficient waiting
>> >
>> > Notable in this release is HBASE-10501, which changes how regions are
>> > distributed for an initially empty table.
>> >
>> > The list of changes is also available here:
>> >
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325845
>> >
>> > Here're the jenkins runs for this RC:
>> > https://builds.apache.org/job/HBase-0.94.17/15/ and
>> > https://builds.apache.org/job/HBase-0.94.17-security/5/
>> >
>> > Please try out the RC, check out the doc, take it for a spin, etc, and
>> > vote +1/-1 by EOD February 24th on whether we should release this as
>> > 0.94.17.
>> >
>> > As before, I realize that release testing is like pulling teeth, but
>> > please give this is quick run, so we can get these bug fixes out there.
>> >
>> > Thanks.
>> >
>> > -- Lars
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message