flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <balassi.mar...@gmail.com>
Subject Re: Problem with deploying a flink topology in a remote execution environment
Date Mon, 10 Nov 2014 15:51:59 GMT
Hey Attila,

Thanks for the detailed bug report. I've picked up the issue and fixed the
build for the streaming example packages. [1]
There is an outstanding issue with the most likely the class loading when
executing streaming code remotely. I've started working on it, Robert was
kind enough to give me some guidance on it.

[1]
https://github.com/mbalassi/incubator-flink/commit/2be50b644807aeead825f512e13b21702404560f

Getting back to you as soon as it is fixed,

Marton

On Wed, Nov 5, 2014 at 5:08 PM, axt <axt@load.hu> wrote:

>  Hi Marton! Sorry for my late response.
>
> My classes are serializable, thats not what causing the error.
>
> I've managed to narrow the problem (to the official examples, without
> using any own code):
>
> I can run the wordcount example (with input supplied) and the twitter
> example, locally.
> Unfortunatelly  when I try to submit them to the flink-on-yarn instance,
> in both cases, I get the ClassNotFound errors. Stack traces are here:
> http://pastebin.com/MMLaNTcv
>
> I've tried it with two different hadoop installs.
>
> I'm starting flink-on-yarn with this command:
> HADOOP_HOME=/usr/lib/hadoop
> /opt/flink-yarn-0.7.0-incubating/bin/yarn-session.sh -n 2 -jm 1024 -tm 4096
> -s 4
>
> Can you give me the versions what you are using in your test environments?
> Do you have any specific configuration options?
>
> Maybe any ideas how should I debug this problem?
>
> Just a note: the
> "org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator.groupBy([I)Lorg/apache/flink/streaming/api/datastream/GroupedDataStream;"
> errors have disappeared, it was caused just by an inconsistent build.
>
> Greetings,
>   Attila
>
>
>
> On 2014-11-03 16:57, Márton Balassi wrote:
>
>  Hey Attila,
>
>  Thanks for trying out streaming!
>
>  As for your issues:
>
>     * Your simple topology: make sure that your user defined functions
> (e.g. ItemsKafkaSource)  and the classes containing them are
> serializable, so that they can be shipped. For anything more specific if
> you can give send over your code to me I am more than happy to have a look
> at it.
>    * Streaming wordcount not working without provided dataset: Yes, thanks
> for reporting it is my fault - which was known, but now I also have a JIRA
> [1] for it. :) However if you give it some input it should work.
>    * Can you run the Twitter example locally by the way?
>
>  [1] https://issues.apache.org/jira/browse/FLINK-1204
>
>  Cheers,
>
>  Marton
>
>
> On Mon, Nov 3, 2014 at 2:53 PM, axt <axt@load.hu> wrote:
>
>> ClassNotFoundException:
>> com.gravityrd.flink.ItemsKafkaSource
>>
>
>
>
>

Mime
View raw message