flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lydia Ickler <ickle...@googlemail.com>
Subject Re: data flow example on cluster
Date Sat, 03 Oct 2015 06:41:21 GMT
Thanks, Till!
I used the ALS from FlinkML and it works :)

Best regards,
Lydia

> Am 02.10.2015 um 14:14 schrieb Till Rohrmann <trohrmann@apache.org>:
> 
> Hi Lydia,
> 
> I think the APIs of the versions 0.8-incubating-SNAPSHOT and 0.10-SNAPSHOT are not compatible.
Thus, it’s not just simply setting the dependencies to 0.10-SNAPSHOT. You also have to fix
the API changes. This might not be trivial. Therefore, I’d recommend you to simply use the
ALS implementation which you can find in FlinkML.
> 
> Cheers,
> Till
> 
> 
> On Fri, Oct 2, 2015 at 2:11 PM, Robert Metzger <rmetzger@apache.org <mailto:rmetzger@apache.org>>
wrote:
> Lydia,
> can you check the log of Flink installed on the cluster? During startup, it is writing
the exact commit your 0.10-SNAPSHOT is based on.
> 
> I would recommend to check out exactly that commit locally and then build Flink locally.
After that, you can rebuild your jobs jar again.
> With that method, there is certainly no version mismatch.
> 
> (Thats one of the reasons why using SNAPSHOT versions on such environments is not recommended)
> 
> 
> 
> 
> On Fri, Oct 2, 2015 at 2:05 PM, Stefano Bortoli <s.bortoli@gmail.com <mailto:s.bortoli@gmail.com>>
wrote:
> I had problems running a flink job with maven, probably there is some issue of classloading.
For me worked to run a simple java command with the uberjar. So I build the jar using maven,
and then run it this way
> 
> java -Xmx2g -cp target/youruberjar.jar yourclass arg1 arg2
> 
> hope it helps,
> Stefano
> 
> 2015-10-02 12:21 GMT+02:00 Lydia Ickler <icklerly@googlemail.com <mailto:icklerly@googlemail.com>>:
> Hi,
> 
> I did not create anything by myself.
> I just downloaded the files from here: https://github.com/tillrohrmann/flink-perf <https://github.com/tillrohrmann/flink-perf>
> 
> And then executed mvn clean install -DskipTests
> 
> Then I opened the project within IntelliJ and there it works fine.
> Then I exported it to the cluster that runs with 0.10-SNAPSHOT.
> 
> 
>> Am 02.10.2015 um 12:15 schrieb Stephan Ewen <sewen@apache.org <mailto:sewen@apache.org>>:
>> 
>> @Lydia  Did you create your POM files for your job with an 0.8.x quickstart?
>> 
>> Can you try to simply re-create your project's POM files with a new quickstart?
>> 
>> I think that the POMS between 0.8-incubating-SNAPSHOT and 0.10-SNAPSHOT may not be
quite compatible any more...
>> 
>> On Fri, Oct 2, 2015 at 12:07 PM, Robert Metzger <rmetzger@apache.org <mailto:rmetzger@apache.org>>
wrote:
>> Are you relying on a feature only available in 0.10-SNAPSHOT?
>> Otherwise, I would recommend to use the latest stable release (0.9.1) for your flink
job and on the cluster.
>> 
>> On Fri, Oct 2, 2015 at 11:55 AM, Lydia Ickler <icklerly@googlemail.com <mailto:icklerly@googlemail.com>>
wrote:
>> Hi,
>> 
>> but inside the pom of flunk-job is the flink version set to 0.8
>> 
>> 		<flink.version>0.8-incubating-SNAPSHOT</flink.version>
>> 
>> how can I change it to the newest?
>> 		<flink.version>0.10-SNAPSHOT</flink.version>
>> Is not working
>> 
>>> Am 02.10.2015 um 11:48 schrieb Robert Metzger <rmetzger@apache.org <mailto:rmetzger@apache.org>>:
>>> 
>>> I think there is a version mismatch between the Flink version you've used to
compile your job and the Flink version installed on the cluster.
>>> 
>>> Maven automagically pulls newer 0.10-SNAPSHOT versions every time you're building
your job.
>>> 
>>> On Fri, Oct 2, 2015 at 11:45 AM, Lydia Ickler <icklerly@googlemail.com <mailto:icklerly@googlemail.com>>
wrote:
>>> Hi Till,
>>> I want to execute your Matrix Completion program „ALSJoin“.
>>> 
>>> Locally it works perfect.
>>> Now I want to execute it on the cluster with:
>>> 
>>> run -c com.github.projectflink.als.ALSJoin -cp /tmp/icklerly/flink-jobs-0.1-SNAPSHOT.jar
0 2 0.001 10 1 1
>>> 
>>> but I get the following error:
>>> java.lang.NoSuchMethodError: org.apache.flink.api.scala.typeutils.CaseClassTypeInfo.<init>(Ljava/lang/Class;Lscala/collection/Seq;Lscala/collection/Seq;)V
>>> 
>>> I guess something like the flink-scala-0.10-SNAPSHOT.jar is missing.
>>> How can I add that to the path?
>>> 
>>> Best regards,
>>> Lydia
>>> 
>>> 
>> 
>> 
>> 
> 
> 
> 
> 


Mime
View raw message