flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "LINZ, Arnaud" <AL...@bouyguestelecom.fr>
Subject Flink Java 8 problem (no lambda, simple code)
Date Fri, 24 Apr 2015 09:13:42 GMT
Hi,

I have the following simple code that works well in Java 7 :

        final ExecutionEnvironment cluster = ExecutionEnvironment.createLocalEnvironment();
        final DataSet<String> textFile = cluster.readTextFile(MiscTools.chercher("jeuDeDonnees.txt"));
        final DataSet<Tuple2<String, Integer>> words = textFile
            .flatMap(new FlatMapFunction<String, Tuple2<String, Integer>>() {
                @Override
                public void flatMap(String ligne, Collector<Tuple2<String, Integer>>
out) {
                    for (final String word : ligne.split("\\s")) {
                        out.collect(new Tuple2<String, Integer>(word, 1));
                    }
                }
            });
        final DataSet<Tuple2<String, Integer>> wordsCount = words.groupBy(0).sum(1);
        wordsCount.print();
        cluster.execute("testFlink");

When compiled in Java 8 and executed (Oracle JDK or Eclipse JDT compiler, same result) I have
the following stack trace (under eclipse or with maven test) :

java.lang.IllegalArgumentException: null
       at org.apache.flink.shaded.org.objectweb.asm.ClassReader.<init>(Unknown Source)
       at org.apache.flink.shaded.org.objectweb.asm.ClassReader.<init>(Unknown Source)
       at org.apache.flink.shaded.org.objectweb.asm.ClassReader.<init>(Unknown Source)
       at org.apache.flink.api.java.ClosureCleaner.getClassReader(ClosureCleaner.java:40)
       at org.apache.flink.api.java.ClosureCleaner.cleanThis0(ClosureCleaner.java:67)
       at org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:54)
       at org.apache.flink.api.java.DataSet.clean(DataSet.java:185)
       at org.apache.flink.api.java.DataSet.flatMap(DataSet.java:266)


Any idea why ?

Here is my pom.xml :

       <build>
             <plugins>
                    <!-- Surefire plugin -->
                    <plugin>
                           <groupId>org.apache.maven.plugins</groupId>
                           <artifactId>maven-surefire-plugin</artifactId>
                           <version>2.15</version>
                           <configuration>
                                  <jvm>${env.JAVA_HOME}/bin/java</jvm>
                                  <forkMode>once</forkMode>
                           </configuration>
                    </plugin>


                    <plugin>
                           <groupId>org.apache.maven.plugins</groupId>
                           <artifactId>maven-compiler-plugin</artifactId>
                           <version>3.1</version>

                           <configuration>
                                  <source>1.8</source>
                                  <target>1.8</target>
                                  <compilerId>jdt</compilerId>
                                  <!-- executable>${env.JAVA_HOME}/bin/javac</executable>
-->
                           </configuration>
                           <dependencies>
                                  <!-- This dependency provides the implementation of compiler
"jdt": -->
                                  <dependency>
                                        <groupId>org.eclipse.tycho</groupId>
                                        <artifactId>tycho-compiler-jdt</artifactId>
                                        <version>0.21.0</version>
                                  </dependency>
                           </dependencies>
                    </plugin>
             </plugins>
             <pluginManagement>
                    <plugins>
                           <!--This plugin's configuration is used to store Eclipse m2e
settings
                                  only. It has no influence on the Maven build itself. -->
                           <plugin>
                                  <groupId>org.eclipse.m2e</groupId>
                                  <artifactId>lifecycle-mapping</artifactId>
                                  <version>1.0.0</version>
                                  <configuration>
                                        <lifecycleMappingMetadata>
                                               <pluginExecutions>
                                                      <pluginExecution>
                                                            <pluginExecutionFilter>
                                                                   <groupId>
                                                                          org.apache.maven.plugins
                                                                   </groupId>
                                                                   <artifactId>
                                                                          maven-compiler-plugin
                                                                   </artifactId>
                                                                   <versionRange>
                                                                          [3.1,)
                                                                   </versionRange>
                                                                   <goals>
                                                                          <goal>compile</goal>
                                                                   </goals>
                                                            </pluginExecutionFilter>
                                                            <action>
                                                                   <ignore></ignore>
                                                            </action>
                                                      </pluginExecution>
                                               </pluginExecutions>
                                        </lifecycleMappingMetadata>
                                  </configuration>
                           </plugin>
                    </plugins>
             </pluginManagement>
       </build>

       <dependencies>
             <dependency>
                    <groupId>junit</groupId>
                    <artifactId>junit</artifactId>
                    <version>4.10</version>
                    <scope>test</scope>
             </dependency>

             <!-- Flink -->
             <dependency>
                    <groupId>org.apache.flink</groupId>
                    <artifactId>flink-java</artifactId>
                    <version>0.9.0-milestone-1</version>
             </dependency>
             <dependency>
                    <groupId>org.apache.flink</groupId>
                    <artifactId>flink-clients</artifactId>
                    <version>0.9.0-milestone-1</version>
             </dependency>


       </dependencies>

A simple replacement of “1.8” by “1.7” in the pom makes the program work.

Regards,
Arnaud

________________________________

L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice
ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation
ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message,
merci de le détruire et d'avertir l'expéditeur.

The integrity of this message cannot be guaranteed on the Internet. The company that sent
this message cannot therefore be held liable for its content nor attachments. Any unauthorized
use or dissemination is prohibited. If you are not the intended recipient of this message,
then please delete it and notify the sender.
Mime
View raw message