flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: asm IllegalArgumentException with 1.0.0
Date Wed, 13 Apr 2016 18:59:38 GMT
Does this problem persist? (It might have been caused by maven caches with
bad artifacts).

The many transitive dependencies you see often come from the connectors -
that is also why we do not put the connectors into the lib folder directly,
so that these libraries are not always part of every Flink program.


On Mon, Mar 14, 2016 at 9:16 PM, Zach Cox <zcox522@gmail.com> wrote:

> Yes pretty much - we use sbt to run the job in a local environment, not
> Intellij, but should be the same thing. We were also seeing that exception
> running unit tests locally. We did not see the exception when assembling a
> fat jar and submitting to a remote Flink cluster.
>
> It seems like the flink-connector-elasticsearch jar should not have shaded
> classes in it. Maybe that jar in maven central was built incorrectly?
>
> We worked around this by just not depending on that elasticsearch
> connector at all, since we wrote our own connector for Elasticsearch 2.x.
>
> -Zach
>
>
> On Mon, Mar 14, 2016 at 2:03 PM Andrew Whitaker <
> andrew.whitaker@braintreepayments.com> wrote:
>
>> We're having the same issue (we also have a dependency on
>> flink-connector-elasticsearch). It's only happening to us in IntelliJ
>> though. Is this the case for you as well?
>>
>> On Thu, Mar 10, 2016 at 3:20 PM, Zach Cox <zcox522@gmail.com> wrote:
>>
>>> After some poking around I noticed
>>> that flink-connector-elasticsearch_2.10-1.0.0.jar contains shaded asm
>>> classes. If I remove that dependency from my project then I do not get the
>>> IllegalArgumentException.
>>>
>>>
>>> On Thu, Mar 10, 2016 at 11:51 AM Zach Cox <zcox522@gmail.com> wrote:
>>>
>>>> Here are the jars on the classpath when I try to run our Flink job in a
>>>> local environment (via `sbt run`):
>>>>
>>>>
>>>> https://gist.githubusercontent.com/zcox/0992aba1c517b51dc879/raw/7136ec034c2beef04bd65de9f125ce3796db511f/gistfile1.txt
>>>>
>>>> There are many transitive dependencies pulled in from internal library
>>>> projects that probably need to be cleaned out. Maybe we are including
>>>> something that conflicts? Or maybe something important is being excluded?
>>>>
>>>> Are the asm classes included in Flink jars in some shaded form?
>>>>
>>>> Thanks,
>>>> Zach
>>>>
>>>>
>>>> On Thu, Mar 10, 2016 at 5:06 AM Stephan Ewen <sewen@apache.org> wrote:
>>>>
>>>>> Dependency shading changed a bit between RC4 and RC5 - maybe a
>>>>> different minor ASM version is now included in the "test" scope.
>>>>>
>>>>> Can you share the dependencies of the problematic project?
>>>>>
>>>>> On Thu, Mar 10, 2016 at 12:26 AM, Zach Cox <zcox522@gmail.com>
wrote:
>>>>>
>>>>>> I also noticed when I try to run this application in a local
>>>>>> environment, I get the same IllegalArgumentException.
>>>>>>
>>>>>> When I assemble this application into a fat jar and run it on a Flink
>>>>>> cluster using the CLI tools, it seems to run fine.
>>>>>>
>>>>>> Maybe my local classpath is missing something that is provided on
the
>>>>>> Flink task managers?
>>>>>>
>>>>>> -Zach
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 9, 2016 at 5:16 PM Zach Cox <zcox522@gmail.com>
wrote:
>>>>>>
>>>>>>> Hi - after upgrading to 1.0.0, I'm getting this exception now
in a
>>>>>>> unit test:
>>>>>>>
>>>>>>>    IllegalArgumentException:   (null:-1)
>>>>>>> org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown
>>>>>>> Source)
>>>>>>> org.apache.flink.shaded.org.objectweb.asm.ClassVisitor.<init>(Unknown
>>>>>>> Source)
>>>>>>>
>>>>>>> org.apache.flink.api.scala.InnerClosureFinder.<init>(ClosureCleaner.scala:279)
>>>>>>>
>>>>>>> org.apache.flink.api.scala.ClosureCleaner$.getInnerClasses(ClosureCleaner.scala:95)
>>>>>>>
>>>>>>> org.apache.flink.api.scala.ClosureCleaner$.clean(ClosureCleaner.scala:115)
>>>>>>>
>>>>>>> org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.scalaClean(StreamExecutionEnvironment.scala:568)
>>>>>>>
>>>>>>> org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.addSource(StreamExecutionEnvironment.scala:498)
>>>>>>>
>>>>>>> The line that causes that exception is just adding
>>>>>>> a FlinkKafkaConsumer08 source.
>>>>>>>
>>>>>>> ClassVisitor [1] seems to throw that IllegalArgumentException
when
>>>>>>> it is not given a valid api version number, but InnerClosureFinder
[2]
>>>>>>> looks fine to me.
>>>>>>>
>>>>>>> Any idea what might be causing this? This unit test worked fine
with
>>>>>>> 1.0.0-rc0 jars.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Zach
>>>>>>>
>>>>>>> [1]
>>>>>>> http://websvn.ow2.org/filedetails.php?repname=asm&path=%2Ftrunk%2Fasm%2Fsrc%2Forg%2Fobjectweb%2Fasm%2FClassVisitor.java
>>>>>>> [2]
>>>>>>> https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/ClosureCleaner.scala#L279
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>
>>
>> --
>> Andrew Whitaker | andrew.whitaker@braintreepayments.com
>> --
>> Note: this information is confidential. It is prohibited to share, post
>> online or otherwise publicize without Braintree's prior written consent.
>>
>

Mime
View raw message