flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chesnay Schepler <ches...@apache.org>
Subject Re: GetExecutionPlan fails with IllegalArgumentException in Comparator
Date Mon, 29 Jan 2018 15:05:06 GMT
@Julian Could you provide an example job to reproduce this? (Can also be 
shared privately)

On 23.01.2018 18:05, Fabian Hueske wrote:
> Great! Thank you :-)
>
> 2018-01-23 17:58 GMT+01:00 Bauss, Julian <Julian.Bauss@bonprix.net 
> <mailto:Julian.Bauss@bonprix.net>>:
>
>     Hi Fabian,
>
>     sure!
>
>     See: https://issues.apache.org/jira/browse/FLINK-8498
>     <https://issues.apache.org/jira/browse/FLINK-8498>
>
>     Best Regards,
>
>     Julian
>
>     *Von:*Fabian Hueske [mailto:fhueske@gmail.com
>     <mailto:fhueske@gmail.com>]
>     *Gesendet:* Montag, 22. Januar 2018 14:42
>     *An:* Bauss, Julian <Julian.Bauss@bonprix.net
>     <mailto:Julian.Bauss@bonprix.net>>
>     *Cc:* user@flink.apache.org <mailto:user@flink.apache.org>
>     *Betreff:* Re: GetExecutionPlan fails with
>     IllegalArgumentException in Comparator
>
>     Hi Julian,
>
>     I searched for the issue in JIRA [1] but did not find a
>     corresponding issue.
>
>     Could you open an issue for this bug?
>
>     Thank you,
>
>     Fabian
>
>     [1] https://issues.apache.org/jira/projects/FLINK/summary
>     <https://issues.apache.org/jira/projects/FLINK/summary>
>
>     2018-01-22 14:11 GMT+01:00 Bauss, Julian <Julian.Bauss@bonprix.net
>     <mailto:Julian.Bauss@bonprix.net>>:
>
>         Hello everybody,
>
>         we‘re currently encountering an exception while generating an
>         ExecutionGraph JSON in Flink v1.3.2.
>
>         Actually executing the job does not cause an exception and
>         everything works as inteded.
>
>         This happens since we started adding side-outputs to many of
>         our operators.
>
>         Is this already a known bug?
>
>         Below is the stacktrace. The problem seems to be a contract
>         violation in the comparator implementation.
>
>         org.apache.flink.client.program.ProgramInvocationException:
>         The main method caused an error.
>
>                 at
>         org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:545)
>
>                 at
>         org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:419)
>
>                 at
>         org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:381)
>
>                 at
>         org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:838)
>
>                 at
>         org.apache.flink.client.CliFrontend.run(CliFrontend.java:259)
>
>                 at
>         org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1086)
>
>                 at
>         org.apache.flink.client.CliFrontend$2.call(CliFrontend.java:1133)
>
>                 at
>         org.apache.flink.client.CliFrontend$2.call(CliFrontend.java:1130)
>
>                 at
>         org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
>
>                 at java.security.AccessController.doPrivileged(Native
>         Method)
>
>                 at javax.security.auth.Subject.doAs(Subject.java:422)
>
>                 at
>         org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
>
>                 at
>         org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
>
>                 at
>         org.apache.flink.client.CliFrontend.main(CliFrontend.java:1130)
>
>         Caused by: java.lang.RuntimeException: JSON plan creation failed
>
>                 at
>         org.apache.flink.streaming.api.graph.StreamGraph.getStreamingPlanAsJSON(StreamGraph.java:668)
>
>                 at
>         org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionPlan(StreamExecutionEnvironment.java:1538)
>
>                 at com.example.Main.main(Main.java:262)
>
>                 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>         Method)
>
>                 at
>         sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
>                 at
>         sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
>                 at java.lang.reflect.Method.invoke(Method.java:498)
>
>                 at
>         org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:528)
>
>                 ... 13 more
>
>         Caused by: java.lang.IllegalArgumentException: Comparison
>         method violates its general contract!
>
>                 at java.util.TimSort.mergeLo(TimSort.java:777)
>
>                 at java.util.TimSort.mergeAt(TimSort.java:514)
>
>                 at java.util.TimSort.mergeCollapse(TimSort.java:441)
>
>                 at java.util.TimSort.sort(TimSort.java:245)
>
>                 at java.util.Arrays.sort(Arrays.java:1512)
>
>                 at java.util.ArrayList.sort(ArrayList.java:1454)
>
>                 at java.util.Collections.sort(Collections.java:175)
>
>                 at
>         org.apache.flink.streaming.api.graph.JSONGenerator.getJSON(JSONGenerator.java:60)
>
>                 at
>         org.apache.flink.streaming.api.graph.StreamGraph.getStreamingPlanAsJSON(StreamGraph.java:665)
>
>                 ... 21 more
>
>         Best Regards,
>
>         Julian
>
>
>
>
>                     **************************************************************************************************************
>
>
>                     bonprixHandelsgesellschaft mbH
>                     Sitz der Gesellschaft: Hamburg
>
>                     Geschäftsführung:
>                     Markus Fuchshofen
>                     Dr. Richard Gottwald
>                     Dr. Kai Heck
>                     Rien Jansen
>                     Beiratsvorsitzender: Dr. Marcus Ackermann
>
>                     Handelsregister AG Hamburg HR B 36 455
>
>                     Adresse:
>
>                     bonprix Handelsgesellschaft mbH
>
>                     Haldesdorfer Str. 61
>                     <https://maps.google.com/?q=Haldesdorfer+Str.+61+%0D+22179+Hamburg&entry=gmail&source=g>
>                     22179 Hamburg
>
>                     Diese E-Mail enthält vertrauliche und/oder
>                     rechtlich geschützte Informationen.
>                     Wenn Sie nicht der richtige Adressat sind oder
>                     diese E-Mail irrtümlich erhalten haben,
>                     informieren Sie bitte sofort den Absender und
>                     vernichten Sie diese Mail.
>                     Das unerlaubte Kopieren sowie die unbefugte
>                     Weitergabe dieser E-Mail ist nicht gestattet.
>
>                     This e-mail may contain confidential and/or
>                     privileged information.
>                     If you are not the intended recipient (or have
>                     received the e-mail in error)
>                     please notify the sender immediately and delete
>                     this e-mail. Any unauthorized copying,
>                     disclosure or distribution of the material in this
>                     e-mail is strictly forbidden.
>
>                     **************************************************************************************************************
>
>
>
>
>
>                 **************************************************************************************************************
>
>
>                 bonprixHandelsgesellschaft mbH
>                 Sitz der Gesellschaft: Hamburg
>
>                 Geschäftsführung:
>                 Markus Fuchshofen
>                 Dr. Richard Gottwald
>                 Dr. Kai Heck
>                 Rien Jansen
>                 Beiratsvorsitzender: Dr. Marcus Ackermann
>
>                 Handelsregister AG Hamburg HR B 36 455
>
>                 Adresse:
>
>                 bonprix Handelsgesellschaft mbH
>
>                 Haldesdorfer Str. 61
>                 <https://maps.google.com/?q=Haldesdorfer+Str.+61+%0D+22179+Hamburg&entry=gmail&source=g>
>                 22179 Hamburg
>
>                 Diese E-Mail enthält vertrauliche und/oder rechtlich
>                 geschützte Informationen.
>                 Wenn Sie nicht der richtige Adressat sind oder diese
>                 E-Mail irrtümlich erhalten haben,
>                 informieren Sie bitte sofort den Absender und
>                 vernichten Sie diese Mail.
>                 Das unerlaubte Kopieren sowie die unbefugte Weitergabe
>                 dieser E-Mail ist nicht gestattet.
>
>                 This e-mail may contain confidential and/or privileged
>                 information.
>                 If you are not the intended recipient (or have
>                 received the e-mail in error)
>                 please notify the sender immediately and delete this
>                 e-mail. Any unauthorized copying,
>                 disclosure or distribution of the material in this
>                 e-mail is strictly forbidden.
>
>                 **************************************************************************************************************
>
>
>


Mime
View raw message