flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From axt <...@load.hu>
Subject Re: Problem with deploying a flink topology in a remote execution environment
Date Wed, 05 Nov 2014 16:08:44 GMT
Hi Marton! Sorry for my late response.

My classes are serializable, thats not what causing the error.

I've managed to narrow the problem (to the official examples, without
using any own code):

I can run the wordcount example (with input supplied) and the twitter
example, locally.
Unfortunatelly  when I try to submit them to the flink-on-yarn instance,
in both cases, I get the ClassNotFound errors. Stack traces are here:
http://pastebin.com/MMLaNTcv

I've tried it with two different hadoop installs.

I'm starting flink-on-yarn with this command:
HADOOP_HOME=/usr/lib/hadoop
/opt/flink-yarn-0.7.0-incubating/bin/yarn-session.sh -n 2 -jm 1024 -tm
4096 -s 4

Can you give me the versions what you are using in your test
environments? Do you have any specific configuration options?

Maybe any ideas how should I debug this problem?

Just a note: the
"org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator.groupBy([I)Lorg/apache/flink/streaming/api/datastream/GroupedDataStream;"
errors have disappeared, it was caused just by an inconsistent build.

Greetings,
  Attila


On 2014-11-03 16:57, Márton Balassi wrote:
> Hey Attila,
>
> Thanks for trying out streaming!
>
> As for your issues:
>
>    * Your simple topology: make sure that your user defined functions
> (e.g. ItemsKafkaSource)  and the classes containing them are
> serializable, so that they can be shipped. For anything more specific
> if you can give send over your code to me I am more than happy to have
> a look at it.
>    * Streaming wordcount not working without provided dataset: Yes,
> thanks for reporting it is my fault - which was known, but now I also
> have a JIRA [1] for it. :) However if you give it some input it should
> work.
>    * Can you run the Twitter example locally by the way?
>
> [1] https://issues.apache.org/jira/browse/FLINK-1204
>
> Cheers,
>
> Marton
>
>
> On Mon, Nov 3, 2014 at 2:53 PM, axt <axt@load.hu <mailto:axt@load.hu>>
> wrote:
>
>     ClassNotFoundException:
>     com.gravityrd.flink.ItemsKafkaSource
>
>
>


Mime
View raw message