hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <andrew.purt...@gmail.com>
Subject Re: [VOTE] First release candidate for HBase 2.4.0 (RC1) is available
Date Thu, 10 Dec 2020 01:23:07 GMT
ICYMI, the nightly build passes because the JDK version there has three components “11.0.6”
and yours fails because your JDK version has four components (“11.0.9.1”). 

As this is a release vote you do not need to withdraw the veto, it does not block, but I would
ask that you fix your local environment to conform the the limitations of Jetty as transitively
pulled in via that optional profile and retest. This is not an HBase issue and it should not
be relevant in voting. (IMHO)


> On Dec 9, 2020, at 5:10 PM, Andrew Purtell <andrew.purtell@gmail.com> wrote:
> 
> Nick,
> 
> Given the nature of this issue I’d ask you to try Duo’s suggestion and if an earlier
version of Hadoop 3 succeeds that this be sufficient this time around. 
> 
> All,
> 
> I will start a DISCUSS thread as follow up as to what should be considered required and
veto worthy for a RC and what should not, with regard to optional build profiles. In my opinion
‘required‘ should be defined as what is enabled by default in the release build, and ‘optional’
is everything else until we agree to specifically include one or more optional build profiles
to the required list. 
> 
> 
>> On Dec 9, 2020, at 4:31 PM, 张铎 <palomino219@gmail.com> wrote:
>> 
>> OK, I think the problem is a bug in jetty 9.3, the JavaVersion
>> implementation for 9.3 and 9.4 are completely different, there is no
>> problem to parse 11.0.9.1 for jetty 9.4 but for jetty 9.3, it can only pass
>> version with two dots, i.e, 11.0.9.
>> 
>> I think you could add -Dhadoop-three.version to set the hadoop 3 version
>> explicitly to a newer version which uses jetty 9.4 to solve the problem,
>> IIRC all the newest release for each active release line has upgrade to
>> jetty 9.4 and that's why we need to shade jetty as jetty 9.3 and 9.4 are
>> incompatible.
>> 
>> Thanks.
>> 
>> 张铎(Duo Zhang) <palomino219@gmail.com> 于2020年12月10日周四 上午8:21写道:
>> 
>>> On nightly jdk11 build the jdk version is
>>> 
>>> AdoptOpenJDK-11.0.6+10
>>> 
>>> Andrew Purtell <apurtell@apache.org> 于2020年12月10日周四 上午7:21写道:
>>> 
>>>> Let me rephrase.
>>>> 
>>>> I'm all for testing the optional configurations. I'm less supportive of
>>>> vetoing releases when an optional configuration has some issue due to a
>>>> third party component. I would like to see us veto only on the required
>>>> configurations, and otherwise file JIRAs to fix up the nits on the
>>>> optional
>>>> ones.
>>>> 
>>>> 
>>>> On Wed, Dec 9, 2020 at 3:19 PM Andrew Purtell <apurtell@apache.org>
>>>> wrote:
>>>> 
>>>>>> parseJDK9:71, JavaVersion (org.eclipse.jetty.util)
>>>>> 
>>>>> So unless I am mistaken, some Jetty utility class is not able to parse
>>>> the
>>>>> version string of your particular JDK/JRE.
>>>>> 
>>>>> We can try to downgrade Jetty but I am not sure in general how we are
>>>>> supposed to take on the risk of third party dependencies doing the wrong
>>>>> thing in an optional configuration. I for one do not want to deal with
a
>>>>> combinatorial explosion of transitive dependencies when releasing.
>>>>> 
>>>>> 
>>>>> On Wed, Dec 9, 2020 at 2:41 PM Nick Dimiduk <ndimiduk@apache.org>
>>>> wrote:
>>>>> 
>>>>>> This is coming out of Jetty + Hadoop. This build has a regression
in
>>>> our
>>>>>> JDK11 support. Or perhaps there's a regression in the upstream Hadoop
>>>>>> against which JDK11 builds.
>>>>>> 
>>>>>> I'm afraid I must vote -1 until we can sort out the issue. I'd
>>>> appreciate
>>>>>> it if someone else can attempt to repro, help ensure it's not just
me.
>>>>>> 
>>>>>> Thanks,
>>>>>> Nick
>>>>>> 
>>>>>> (Apologies for the crude stack trace; this is copied out of an attached
>>>>>> debugger)
>>>>>> 
>>>>>> parseJDK9:71, JavaVersion (org.eclipse.jetty.util)
>>>>>> parse:49, JavaVersion (org.eclipse.jetty.util)
>>>>>> <clinit>:43, JavaVersion (org.eclipse.jetty.util)
>>>>>> findAndFilterContainerPaths:185, WebInfConfiguration
>>>>>> (org.eclipse.jetty.webapp)
>>>>>> preConfigure:155, WebInfConfiguration (org.eclipse.jetty.webapp)
>>>>>> preConfigure:485, WebAppContext (org.eclipse.jetty.webapp)
>>>>>> doStart:521, WebAppContext (org.eclipse.jetty.webapp)
>>>>>> start:68, AbstractLifeCycle (org.eclipse.jetty.util.component)
>>>>>> start:131, ContainerLifeCycle (org.eclipse.jetty.util.component)
>>>>>> doStart:113, ContainerLifeCycle (org.eclipse.jetty.util.component)
>>>>>> doStart:61, AbstractHandler (org.eclipse.jetty.server.handler)
>>>>>> start:68, AbstractLifeCycle (org.eclipse.jetty.util.component)
>>>>>> start:131, ContainerLifeCycle (org.eclipse.jetty.util.component)
>>>>>> start:427, Server (org.eclipse.jetty.server)
>>>>>> doStart:105, ContainerLifeCycle (org.eclipse.jetty.util.component)
>>>>>> doStart:61, AbstractHandler (org.eclipse.jetty.server.handler)
>>>>>> doStart:394, Server (org.eclipse.jetty.server)
>>>>>> start:68, AbstractLifeCycle (org.eclipse.jetty.util.component)
>>>>>> start:1140, HttpServer2 (org.apache.hadoop.http)
>>>>>> start:177, NameNodeHttpServer (org.apache.hadoop.hdfs.server.namenode)
>>>>>> startHttpServer:872, NameNode (org.apache.hadoop.hdfs.server.namenode)
>>>>>> initialize:694, NameNode (org.apache.hadoop.hdfs.server.namenode)
>>>>>> <init>:940, NameNode (org.apache.hadoop.hdfs.server.namenode)
>>>>>> <init>:913, NameNode (org.apache.hadoop.hdfs.server.namenode)
>>>>>> createNameNode:1646, NameNode (org.apache.hadoop.hdfs.server.namenode)
>>>>>> createNameNode:1314, MiniDFSCluster (org.apache.hadoop.hdfs)
>>>>>> configureNameService:1083, MiniDFSCluster (org.apache.hadoop.hdfs)
>>>>>> createNameNodesAndSetConf:958, MiniDFSCluster (org.apache.hadoop.hdfs)
>>>>>> initMiniDFSCluster:890, MiniDFSCluster (org.apache.hadoop.hdfs)
>>>>>> <init>:518, MiniDFSCluster (org.apache.hadoop.hdfs)
>>>>>> build:477, MiniDFSCluster$Builder (org.apache.hadoop.hdfs)
>>>>>> startMiniDFSCluster:108, AsyncFSTestBase
>>>>>> (org.apache.hadoop.hbase.io.asyncfs)
>>>>>> setUp:87, TestFanOutOneBlockAsyncDFSOutput
>>>>>> (org.apache.hadoop.hbase.io.asyncfs)
>>>>>> invoke0:-1, NativeMethodAccessorImpl (jdk.internal.reflect)
>>>>>> invoke:62, NativeMethodAccessorImpl (jdk.internal.reflect)
>>>>>> invoke:43, DelegatingMethodAccessorImpl (jdk.internal.reflect)
>>>>>> invoke:566, Method (java.lang.reflect)
>>>>>> runReflectiveCall:59, FrameworkMethod$1 (org.junit.runners.model)
>>>>>> run:12, ReflectiveCallable (org.junit.internal.runners.model)
>>>>>> invokeExplosively:56, FrameworkMethod (org.junit.runners.model)
>>>>>> invokeMethod:33, RunBefores (org.junit.internal.runners.statements)
>>>>>> evaluate:24, RunBefores (org.junit.internal.runners.statements)
>>>>>> evaluate:27, RunAfters (org.junit.internal.runners.statements)
>>>>>> evaluate:38, SystemExitRule$1 (org.apache.hadoop.hbase)
>>>>>> call:288, FailOnTimeout$CallableStatement
>>>>>> (org.junit.internal.runners.statements)
>>>>>> call:282, FailOnTimeout$CallableStatement
>>>>>> (org.junit.internal.runners.statements)
>>>>>> run:264, FutureTask (java.util.concurrent)
>>>>>> run:834, Thread (java.lang)
>>>>>> 
>>>>>> On Wed, Dec 9, 2020 at 2:08 PM Nick Dimiduk <ndimiduk@apache.org>
>>>> wrote:
>>>>>> 
>>>>>>> On Mon, Dec 7, 2020 at 1:51 PM Nick Dimiduk <ndimiduk@apache.org>
>>>>>> wrote:
>>>>>>> 
>>>>>>>> Has anyone successfully built/run this RC with JDK11 and
Hadoop3
>>>>>> profile?
>>>>>>>> I'm seeing test failures locally in the hbase-asyncfs module.
>>>>>>>> Reproducible with:
>>>>>>>> 
>>>>>>>> $
>>>>>>>> 
>>>>>> 
>>>> JAVA_HOME=/Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home
>>>>>>>> mvn clean install -Dhadoop.profile=3.0
>>>>>>>> 
>>>>>> 
>>>> -Dtest=org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput
>>>>>>>> ...
>>>>>>>> [INFO] Running
>>>>>>>> org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput
>>>>>>>> 
>>>>>>>> 
>>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0,
Time
>>>> elapsed:
>>>>>>>> 1.785 s <<< FAILURE! - in
>>>>>>>> org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput
>>>>>>>> 
>>>>>>>> [ERROR]
>>>>>>>> org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput
>>>>>> Time
>>>>>>>> elapsed: 1.775 s  <<< ERROR!
>>>>>>>> 
>>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>>> 
>>>>>>>> 
>>>>>>>>       at
>>>>>>>> org.apache.hadoop.hbase.io
>>>>>> 
>>>> .asyncfs.TestFanOutOneBlockAsyncDFSOutput.setUp(TestFanOutOneBlockAsyncDFSOutput.java:87)
>>>>>>>> 
>>>>>>>> Caused by: java.lang.IllegalArgumentException: Invalid Java
version
>>>>>>>> 11.0.9.1
>>>>>>>> 
>>>>>>>>       at
>>>>>>>> org.apache.hadoop.hbase.io
>>>>>> 
>>>> .asyncfs.TestFanOutOneBlockAsyncDFSOutput.setUp(TestFanOutOneBlockAsyncDFSOutput.java:87)
>>>>>>>> 
>>>>>>> 
>>>>>>> This failure is not isolated to macOS. I ran this build on an
Ubuntu
>>>> VM
>>>>>>> with the same AdoptOpenJDK 11.0.9.1. Why don't we see this in
>>>> Jenkins?
>>>>>>> 
>>>>>>> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
>>>> elapsed:
>>>>>>> 0.011 s <<< FAILURE! - in
>>>>>>> org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog
>>>>>>> 
>>>>>>> [ERROR] org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog
>>>>>>> Time elapsed: 0.003 s  <<< ERROR!
>>>>>>> 
>>>>>>> java.lang.ExceptionInInitializerError
>>>>>>> 
>>>>>>>       at
>>>>>>> 
>>>>>> 
>>>> org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog.setUpBeforeClass(TestAsyncProtobufLog.java:56)
>>>>>>> 
>>>>>>>       Caused by: java.lang.IllegalArgumentException: Invalid
Java
>>>>>> version
>>>>>>> 11.0.9.1
>>>>>>>       at
>>>>>>> 
>>>>>> 
>>>> org.apache.hadoop.hbase.regionserver.wal.TestAsyncProtobufLog.setUpBeforeClass(TestAsyncProtobufLog.java:56)
>>>>>>> 
>>>>>>> On Thu, Dec 3, 2020 at 4:05 PM Andrew Purtell <apurtell@apache.org>
>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Please vote on this Apache hbase release candidate, hbase-2.4.0RC1
>>>>>>>>> 
>>>>>>>>> The VOTE will remain open for at least 72 hours.
>>>>>>>>> 
>>>>>>>>> [ ] +1 Release this package as Apache hbase 2.4.0
>>>>>>>>> [ ] -1 Do not release this package because ...
>>>>>>>>> 
>>>>>>>>> The tag to be voted on is 2.4.0RC1:
>>>>>>>>> 
>>>>>>>>>   https://github.com/apache/hbase/tree/2.4.0RC1
>>>>>>>>> 
>>>>>>>>> The release files, including signatures, digests, as
well as
>>>>>> CHANGES.md
>>>>>>>>> and RELEASENOTES.md included in this RC can be found
at:
>>>>>>>>> 
>>>>>>>>>   https://dist.apache.org/repos/dist/dev/hbase/2.4.0RC1/
>>>>>>>>> 
>>>>>>>>> Customarily Maven artifacts would be available in a staging
>>>>>> repository.
>>>>>>>>> Unfortunately I was forced to terminate the Maven deploy
step after
>>>>>>>>> the upload ran for more than four hours and my build
equipment
>>>>>>>>> needed to be relocated, with loss of network connectivity.
This RC
>>>> has
>>>>>>>>> been delayed long enough. A temporary Maven repository
is not a
>>>>>>>>> requirement for a vote. I will retry Maven deploy tomorrow.
I can
>>>>>>>>> promise the artifacts for this RC will be staged in Apache
Nexus
>>>> and
>>>>>>>>> ready for release well ahead of the earliest possible
time this
>>>> vote
>>>>>>>>> can complete.
>>>>>>>>> 
>>>>>>>>> Artifacts were signed with the apurtell@apache.org key
which can
>>>> be
>>>>>>>>> found
>>>>>>>>> in:
>>>>>>>>> 
>>>>>>>>>   https://dist.apache.org/repos/dist/release/hbase/KEYS
>>>>>>>>> 
>>>>>>>>> The API compatibility report for this RC can be found
at:
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>> 
>>>> https://dist.apache.org/repos/dist/dev/hbase/2.4.0RC1/api_compare_2.4.0RC1_to_2.3.0.html
>>>>>>>>> 
>>>>>>>>> The changes are mostly added methods, which conform to
the
>>>>>> compatibility
>>>>>>>>> guidelines for a new minor release. There is one change
to the
>>>> public
>>>>>>>>> Region interface that alters the return type of a method.
This is
>>>>>>>>> equivalent to a removal then addition and can be a binary
>>>>>> compatibility
>>>>>>>>> problem. However to your RM's eye the change looks intentional
and
>>>> is
>>>>>>>>> part of an API improvement project, and a compatibility
method is
>>>> not
>>>>>>>>> possible here because Java doesn't consider return type
when
>>>> deciding
>>>>>> if
>>>>>>>>> one method signature duplicates another.
>>>>>>>>> 
>>>>>>>>> To learn more about Apache HBase, please see
>>>>>>>>> 
>>>>>>>>>   http://hbase.apache.org/
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> Your HBase Release Manager
>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Best regards,
>>>>> Andrew
>>>>> 
>>>>> Words like orphans lost among the crosstalk, meaning torn from truth's
>>>>> decrepit hands
>>>>>  - A23, Crosstalk
>>>>> 
>>>> 
>>>> 
>>>> --
>>>> Best regards,
>>>> Andrew
>>>> 
>>>> Words like orphans lost among the crosstalk, meaning torn from truth's
>>>> decrepit hands
>>>>  - A23, Crosstalk
>>>> 
>>> 

Mime
View raw message