flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@apache.org>
Subject Re: Java 8 JDK Issue
Date Wed, 16 Sep 2015 21:05:59 GMT
Hi,

fixed it... After trying (without success) to build a minimum Java 8
example that fails (a new project extern to Flink), it was clear that
JDK 8 is actually not the problem.

I did some more debugging an figured out, that different class loader
are used here (for the external new Java 8 and original Flink Java 8 test)

> Class<? extends Configuration> implClass = Class.forName(
>         finalClass, false,
>         Thread.currentThread().getContextClassLoader()
> ).asSubclass(Configuration.class);

In the Flink Java 8 case, it was a "MockClassLoader". This reminded me,
that I had a previous Java 7 issue, that I resolved my added a test
annotation.

> @PowerMockIgnore("javax.managment.*")

Now I changed this to

> @PowerMockIgnore("javax.*")

which fixed it. The problem was, that the annotation disabled
MockClassLoader for a too limited "package space" in Java 8.


-Matthias




On 09/16/2015 09:12 PM, Stephan Ewen wrote:
> No clue, sorry.
> 
> The Storm Mailing list may be able to help you, it looks like deep into
> their code...
> 
> On Wed, Sep 16, 2015 at 9:00 PM, Matthias J. Sax <mjsax@apache.org> wrote:
> 
>> Hi,
>>
>> using Java 8 locally, I was able to reproduce the problem. I dug into it
>> and could figure out the following. However, I cannot make any sense out
>> of it. I have no experience with Java security stuff...
>>
>> Anyone?
>>
>> -Matthias
>>
>> ========================================================================
>>
>> In class
>>> javax.security.auth.login.Configuration.getConfiguration()
>>
>> in line 235 to 242 (for Java 7) and
>> in line 232 to 239 (for Java 8)
>>
>> this code
>>
>>> String config_class = null;
>>> config_class = AccessController.doPrivileged
>>>     (new PrivilegedAction<Stritype filter textng>() {
>>>     public String run() {
>>>         return java.security.Security.getProperty
>>>                     ("login.configuration.provider");
>>>     }
>>> });
>>
>> results for "config_class":
>>   Java 7: com.sun.security.auth.login.ConfigFile
>>   Java 8: sun.security.provider.ConfigFile
>>
>> Later this leads to ClassCastException (in line 254 to 257 for Java 7;
>> and line 251 to 254 for Java 8)
>>
>>> Class<? extends Configuration> implClass = Class.forName(
>>>         finalClass, false,
>>>         Thread.currentThread().getContextClassLoader()
>>> ).asSubclass(Configuration.class);
>>
>> The call stack is also somewhat different.
>>
>> Java 7:
>>
>>> Configuration.getConfiguration() line: 235
>>> NIOServerCnxnFactory(ServerCnxnFactory).configureSaslLogin() line: 174
>>> NIOServerCnxnFactory.configure(InetSocketAddress, int) line: 87
>>> zookeeper.clj line: 198
>>> zookeeper.clj line: 197
>>> zookeeper.clj line: 195
>>> zookeeper$mk_inprocess_zookeeper(RestFn).invoke(Object) line: 410
>>> testing.clj line: 123
>>> testing$mk_local_storm_cluster(RestFn).invoke(Object, Object) line: 421
>>> LocalCluster.clj line: 28
>>> LocalCluster.<init>() line: not available
>>> StormWrapperSetupHelperTest.testCreateTopologyContext() line: 190
>>
>> Java 8:
>>
>>> Configuration$2.run() line: 254 [local variables unavailable]
>>> Configuration$2.run() line: 247
>>> AccessController.doPrivileged(PrivilegedExceptionAction<T>) line: not
>> available [native method]
>>> Configuration.getConfiguration() line: 246 [local variables unavailable]
>>> NIOServerCnxnFactory(ServerCnxnFactory).configureSaslLogin() line: 174
>>> NIOServerCnxnFactory.configure(InetSocketAddress, int) line: 87
>>> zookeeper.clj line: 198
>>> zookeeper.clj line: 197
>>> zookeeper.clj line: 195
>>> zookeeper$mk_inprocess_zookeeper(RestFn).invoke(Object) line: 410
>>> testing.clj line: 123
>>> testing$mk_local_storm_cluster(RestFn).invoke(Object, Object) line: 421
>>> LocalCluster.clj line: 28
>>> LocalCluster.<init>() line: not available
>>> StormWrapperSetupHelperTest.testCreateTopologyContext() line: 190
>>
>> ========================================================================
>>
>>
>>
>>
>> On 09/16/2015 12:56 PM, Matthias J. Sax wrote:
>>> Not right now... The ClassCastException occurs deeply inside Storm. I
>>> guess it is a Storm issue.
>>>
>>> I just instantiate a (Storm)LocalCluster and it fails internally.
>>>
>>>> LocalCluster cluster = new LocalCluster();
>>>
>>> Maybe I need to try it locally with Java 8 JDK to be able to debug it...
>>> I will pick up this thread again if I have more information.
>>>
>>> As a fall back, we might want to deactivate this test for Java 8 (if
>>> this is possible).
>>>
>>>
>>> -Matthias
>>>
>>>
>>> On 09/16/2015 12:34 PM, Stephan Ewen wrote:
>>>> Do you know between what classes the exception is? What is the original
>>>> class and what the cast target class?
>>>>
>>>> Are they both the same, and not castable because of different copies
>> being
>>>> loaded from different classloaders, or are they really different types?
>>>>
>>>> On Wed, Sep 16, 2015 at 10:36 AM, Matthias J. Sax <mjsax@apache.org>
>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I just hit a issue with Java 8 JDK. It occurs in a new test I added in
>> a
>>>>> current PR.
>>>>>
>>>>>> Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 8.793
>>>>> sec <<< FAILURE! - in
>>>>>
>> org.apache.flink.stormcompatibility.wrappers.StormWrapperSetupHelperTest
>>>>>>
>>>>>
>> testCreateTopologyContext(org.apache.flink.stormcompatibility.wrappers.StormWrapperSetupHelperTest)
>>>>> Time elapsed: 6.087 sec  <<< ERROR!
>>>>>> java.lang.ClassCastException: class sun.security.provider.ConfigFile
>>>>>>       at java.lang.Class.asSubclass(Class.java:3396)
>>>>>>       at
>>>>> javax.security.auth.login.Configuration$2.run(Configuration.java:254)
>>>>>>       at
>>>>> javax.security.auth.login.Configuration$2.run(Configuration.java:247)
>>>>>>       at java.security.AccessController.doPrivileged(Native Method)
>>>>>>       at
>>>>>
>> javax.security.auth.login.Configuration.getConfiguration(Configuration.java:246)
>>>>>>       at
>>>>>
>> org.apache.storm.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:174)
>>>>>>       at
>>>>>
>> org.apache.storm.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:87)
>>>>>>       at
>>>>>
>> backtype.storm.zookeeper$mk_inprocess_zookeeper$fn__1697$fn__1699.invoke(zookeeper.clj:198)
>>>>>>       at
>>>>>
>> backtype.storm.zookeeper$mk_inprocess_zookeeper$fn__1697.invoke(zookeeper.clj:197)
>>>>>>       at
>>>>>
>> backtype.storm.zookeeper$mk_inprocess_zookeeper.doInvoke(zookeeper.clj:195)
>>>>>>       at clojure.lang.RestFn.invoke(RestFn.java:410)
>>>>>>       at
>>>>> backtype.storm.testing$mk_local_storm_cluster.doInvoke(testing.clj:123)
>>>>>>       at clojure.lang.RestFn.invoke(RestFn.java:421)
>>>>>>       at backtype.storm.LocalCluster$_init.invoke(LocalCluster.clj:28)
>>>>>>       at backtype.storm.LocalCluster.<init>(Unknown Source)
>>>>>>       at
>>>>>
>> org.apache.flink.stormcompatibility.wrappers.StormWrapperSetupHelperTest.testCreateTopologyContext(StormWrapperSetupHelperTest.java:190)
>>>>>
>>>>> Please see here
>>>>>   https://travis-ci.org/mjsax/flink/jobs/80576847
>>>>>
>>>>> I have not clue what the problem might be. Searching on the Internet
>> did
>>>>> not help...
>>>>>
>>>>> Does anyone have an idea what the problem might be?
>>>>>
>>>>>
>>>>> -Matthias
>>>>>
>>>>>
>>>>
>>>
>>
>>
> 


Mime
View raw message