hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: [VOTE] The 2nd hbase 0.94.17 release candidate is available for download
Date Sun, 23 Feb 2014 17:39:02 GMT
To make the read comparison fair, I think '--presplit=' should be used.

Cheers


On Sun, Feb 23, 2014 at 11:15 AM, lars hofhansl <larsh@apache.org> wrote:

> Thanks for the thorough testing.
>
>
> Can you elaborate "page is split according to the MAX_FILESIZE parameter
> until all regions are according to it"?
> As for the 15% slowdown in the random read test... I have looked through
> the issues and did not see anything warranting that, it's also nothing I
> had observed in my testing.
>
> Did you run this with M/R? HBASE-10501 will reduce the default number of
> regions per RegionServer (would be as high as 19 per table before even with
> little data), so maybe you're seeing a bit less parallelism (for small data
> sets).
>
>
> -- Lars
>
>
>
> ________________________________
>  From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
> To: dev <dev@hbase.apache.org>
> Sent: Sunday, February 23, 2014 5:47 AM
> Subject: Re: [VOTE] The 2nd hbase 0.94.17 release candidate is available
> for download
>
>
> Haha ;) Yes. UI. Under the page details in the WebUI, In previous release
> (maybe not 0.94.16 because I did not test it) when clicking split in the
> WebUI, table was split in half. Now, page is split according to the
> MAX_FILESIZE
> parameter until all regions are according to it.
>
> I'm not 100% sure about it so might be wrong. But I'm not sure I have seen
> that before.
>
>
>
> 2014-02-23 8:32 GMT-05:00 Ted Yu <yuzhihong@gmail.com>:
>
> > bq. Click split under UA
> >
> > I guess you meant 'under UI'
> >
> > Cheers
> >
> > On Feb 23, 2014, at 4:47 AM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org>
> > wrote:
> >
> > > Tested, tl.tr: +1 ;)
> > >
> > > All tests are done with Oracle JDK 1.7.0_45 + Hadoop 1.2.1 (Except for
> > the
> > > 0.96.1 test where Hadoop 2.2.0 is used)
> > >
> > > Checked the jars, the signature, the CHANGES.TXT file, the
> documentation
> > =>
> > > Passed.
> > > Checked logs, UI, HBCK, test suite => Passed
> > > Run all Performances tests => Passed. 0.94.17 performances just a bit
> > under
> > > 0.94.16. RandomReadTest 15% slower with 0.94.17. Will retest it. Will
> > post
> > > all the numbers in detail in my blog later today.
> > >
> > > Only concern is again the IntegrationTestLoadAndVerify result where I
> > don't
> > > have the exptected result, but it's the same thing with 0.94.16 and
> > 0.96.1
> > > too. With difference Hadoop version.
> > >
> > >
> > > 0.94.16 + Hadoop 1.2.1:
> > > 14/02/21 21:22:19 INFO mapred.JobClient:
> > > org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify$Counters
> > > 14/02/21 21:22:19 INFO mapred.JobClient:     ROWS_WRITTEN=0
> > > 14/02/21 21:22:19 INFO mapred.JobClient:     REFERENCES_CHECKED=9855584
> > >
> > > 0.94.17 + Hadoop 1.2.1:
> > > 14/02/21 21:31:28 INFO mapred.JobClient:
> > > org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify$Counters
> > > 14/02/21 21:31:28 INFO mapred.JobClient:     ROWS_WRITTEN=0
> > > 14/02/21 21:31:28 INFO mapred.JobClient:     REFERENCES_CHECKED=9855909
> > >
> > > 0.96.1 + Hadoop 2.2.0:
> > > org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify$Counters
> > > REFERENCES_CHECKED=9855588
> > >
> > > Create table, put, scan, compact => Passed.
> > > Offline merged all regions into a single one, hbck, split => Passed.
> > > Rolling restart => Failed once because of jar push issue (my bad),
> > retried
> > > => Passed.
> > > Ran all tests (LoadTestTool, IntegrationTestLoadAndVerify,
> > > HLogPerformanceEvaluation, IntegrationTestBigLinkedList) on a 4 nodes
> > > cluster => Passed.
> > > When we click split in the UA, seems to trigger major compaction, which
> > was
> > > not the case before (Not sure in which version this has been
> introduced)
> > >
> > > Summary:
> > > - IntgrationTestLoadAndVerify results still "strange";
> > > - Split behaviour in the UA seem the have changed.
> > > - +1
> > >
> > > JM
> > > PS: Go Canada Go!
> > >
> > > 2014-02-21 10:01 GMT-05:00 Ted Yu <yuzhihong@gmail.com>:
> > >
> > >> +1 based on the following:
> > >>
> > >> 1. Ran test suite using JDK 1.7 - pass
> > >> 2. compiled Phoenix with locally built 0.94.17 - pass
> > >> 3. checked the documentation
> > >>
> > >> Cheers
> > >>
> > >>
> > >> On Thu, Feb 20, 2014 at 11:18 PM, lars hofhansl <larsh@apache.org>
> > wrote:
> > >>
> > >>> +1
> > >>>
> > >>> - checked documentation and tarball
> > >>>
> > >>> - Ran in local and distributes mode
> > >>> - checked the UI pages
> > >>>
> > >>> - loaded 100m KeyValues
> > >>> - tested scan performance
> > >>>
> > >>> - converted between FAST_DIFF and NONE encoding
> > >>> - tested flushes and compactions
> > >>>
> > >>>
> > >>> -- Lars
> > >>>
> > >>>
> > >>> ________________________________
> > >>> From: lars hofhansl <larsh@apache.org>
> > >>> To: hbase-dev <dev@hbase.apache.org>
> > >>> Sent: Tuesday, February 18, 2014 8:31 PM
> > >>> Subject: [VOTE] The 2nd hbase 0.94.17 release candidate is available
> > for
> > >>> download
> > >>>
> > >>>
> > >>> The 2nd 0.94.17 RC is available for download at
> > >>> http://people.apache.org/~larsh/hbase-0.94.17-rc1/
> > >>> Signed with my code signing key: C7CFE328
> > >>>
> > >>> HBase 0.94.17 is a bug fix release with 35 fixes:
> > >>>    [HBASE-7963] - HBase VerifyReplication not working when security
> > >>> enabled
> > >>>    [HBASE-10249] - TestReplicationSyncUpTool fails because failover
> > >> takes
> > >>> too long
> > >>>    [HBASE-10274] - MiniZookeeperCluster should close ZKDatabase when
> > >>> shutdown ZooKeeperServers
> > >>>    [HBASE-10319] - HLog should roll periodically to allow DN
> > >> decommission
> > >>> to eventually complete.
> > >>>    [HBASE-10320] - Avoid ArrayList.iterator() ExplicitColumnTracker
> > >>>    [HBASE-10335] - AuthFailedException in zookeeper may block
> > >> replication
> > >>> forever
> > >>>    [HBASE-10340] - [BACKPORT] HBASE-9892 Add info port to ServerName
> to
> > >>> support multi instances in a node
> > >>>    [HBASE-10363] - [0.94] TestInputSampler and TestInputSamplerTool
> > fail
> > >>> under hadoop 2.0/23 profiles.
> > >>>    [HBASE-10371] - Compaction creates empty hfile, then selects this
> > >> file
> > >>> for compaction and creates empty hfile and over again
> > >>>    [HBASE-10383] - Secure Bulk Load for 'completebulkload' fails for
> > >>> version 0.94.15
> > >>>    [HBASE-10400] - [hbck] Continue if region dir missing on region
> > merge
> > >>> attempt
> > >>>    [HBASE-10401] - [hbck] perform overlap group merges in parallel
> > >>>    [HBASE-10448] - ZKUtil create and watch methods don't set watch
in
> > >>> some cases
> > >>>    [HBASE-10470] - Import generates huge log file while importing
> large
> > >>> amounts of data
> > >>>    [HBASE-10481] - API Compatibility JDiff script does not properly
> > >>> handle arguments in reverse order
> > >>>    [HBASE-10482] - ReplicationSyncUp doesn't clean up its ZK, needed
> > for
> > >>> tests
> > >>>    [HBASE-10485] - PrefixFilter#filterKeyValue() should perform
> > >> filtering
> > >>> on row key
> > >>>    [HBASE-10489] - TestImportExport fails in 0.94 with Hadoop2
> > >>>    [HBASE-10493] - InclusiveStopFilter#filterKeyValue() should
> perform
> > >>> filtering on row key
> > >>>    [HBASE-10501] - Improve IncreasingToUpperBoundRegionSplitPolicy
to
> > >>> avoid too many regions
> > >>>    [HBASE-10505] - Import.filterKv does not call Filter.filterRowKey
> > >>>    [HBASE-10506] - Fail-fast if client connection is lost before the
> > >> real
> > >>> call be executed in RPC layer
> > >>>    [HBASE-10508] - Backport HBASE-10365 'HBaseFsck should clean up
> > >>> connection properly when repair is completed' to 0.94 and 0.96
> > >>>    [HBASE-10539] - HRegion.addAndGetGlobalMemstoreSize returns
> previous
> > >>> size
> > >>>    [HBASE-10545] - RS Hangs waiting on region to close on shutdown;
> has
> > >>> to timeout before can go down
> > >>>    [HBASE-10546] - Two scanner objects are open for each hbase map
> task
> > >>> but only one scanner object is closed
> > >>>    [HBASE-10551] - Change local mode back to one RS thread by default
> > >>>    [HBASE-10552] -
> > >> HFilePerformanceEvaluation.GaussianRandomReadBenchmark
> > >>> fails sometimes.
> > >>>    [HBASE-10555] - Backport HBASE-8519 to 0.94, Backup master will
> > never
> > >>> come up if primary master dies during initialization
> > >>>    [HBASE-10562] - Fix TestMultiTableInputFormat for Hadoop 2 in 0.94
> > >>>    [HBASE-10212] - New rpc metric: number of active handler
> > >>>    [HBASE-10423] - Report back the message of split or rollback
> failure
> > >>> to the master
> > >>>    [HBASE-10457] - Print corrupted file information in SnapshotInfo
> > tool
> > >>> without -file option
> > >>>    [HBASE-10473] - Add utility for adorning http Context
> > >>>    [HBASE-10480] - TestLogRollPeriod#testWithEdits may fail due to
> > >>> insufficient waiting
> > >>>
> > >>> Notable in this release is HBASE-10501, which changes how regions are
> > >>> distributed for an initially empty table.
> > >>>
> > >>> The list of changes is also available here:
> > >>
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753&version=12325845
> > >>>
> > >>> Here're the jenkins runs for this RC:
> > >>> https://builds.apache.org/job/HBase-0.94.17/15/ and
> > >>> https://builds.apache.org/job/HBase-0.94.17-security/5/
> > >>>
> > >>> Please try out the RC, check out the doc, take it for a spin, etc,
> and
> > >>> vote +1/-1 by EOD February 24th on whether we should release this as
> > >>> 0.94.17.
> > >>>
> > >>> As before, I realize that release testing is like pulling teeth, but
> > >>> please give this is quick run, so we can get these bug fixes out
> there.
> > >>>
> > >>> Thanks.
> > >>>
> > >>> -- Lars
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message