flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trevor Grant <trevor.d.gr...@gmail.com>
Subject Re: Setting up zeppelin with flink
Date Wed, 24 Aug 2016 13:13:12 GMT
Hey Frank,

Saw your post on the Zeppelin list yesterday.  I can look at it later this
morning, but my gut feeling is a ghost Zeppelin daemon is running in the
background and it's local Flink is holding the port 6123. This is fairly
common and would explain the issue.

Idk if you're on linux or windows or whatever, but have you tried rebooting
the machine? (sorry if you said you did higher in the email). Also I very
vaguely remember there is a boot order that matters with Flink and
Zeppelin, like you need to start flink first then zeppelin, or vice verse.
I feel like it is Flink first, then Zeppelin.

Hope that helps, will dig in later if not.

tg





Trevor Grant
Data Scientist
https://github.com/rawkintrevo
http://stackexchange.com/users/3002022/rawkintrevo
http://trevorgrant.org

*"Fortunate is he, who is able to know the causes of things."  -Virgil*


On Wed, Aug 24, 2016 at 6:50 AM, Maximilian Michels <mxm@apache.org> wrote:

> Hi!
>
> There are some people familiar with the Zeppelin integration. CCing
> Till and Trevor. Otherwise, you could also send this to the Zeppelin
> community.
>
> Cheers,
> Max
>
> On Wed, Aug 24, 2016 at 12:58 PM, Frank Dekervel <kervel@gmail.com> wrote:
> > Hello,
> >
> > for reference:
> >
> > i already found out that "connect to existing process" was my error
> here: it
> > means connecting to an existing zeppelin interpreter, not an existing
> flink
> > cluster. After fixing my error, i'm now in the same situation as
> described
> > here:
> >
> > https://stackoverflow.com/questions/38688277/flink-
> zeppelin-not-responding
> >
> > i guess it's more a zeppelin problem than a flink problem tho, as i see
> both
> > interpreter JVM and main zeppelin JVM waiting on thrift input (so it
> seems
> > they are waiting for each other)
> >
> > greetings,
> > Frank
> >
> >
> >
> >
> > On Tue, Aug 23, 2016 at 2:09 PM, Frank Dekervel <kervel@gmail.com>
> wrote:
> >>
> >> Hello,
> >>
> >> I try to set up apache zeppelin with a flink cluster (one jobmanager,
> one
> >> task manager).
> >>
> >> What i did was using the dockerfiles in flink-contrib/docker-flink + the
> >> latest binary release of apache zeppelin with all interpreters:
> >>
> >>
> >> https://github.com/apache/flink/blob/master/flink-contrib/docker-flink/
> Dockerfile
> >> (i changed the flink version to 1.0.3 to match zeppelin's flink version)
> >>
> >> I built another docker image around the latest binary release of
> zeppelin
> >> (with all interpreters), and i reconfigure the flink interpreter:
> >>
> >> connect to existing process
> >> host: jobmanager, port: 6123
> >> i removed all other properties
> >>
> >> when i try to submit a flink job, i get an error state and the following
> >> exception appears in the log (nothing appears in the jobmanager log)
> >>
> >> ERROR [2016-08-23 11:44:57,932] ({Thread-16}
> >> JobProgressPoller.java[run]:54) - Can not get or update progress
> >> org.apache.zeppelin.interpreter.InterpreterException:
> >> org.apache.thrift.transport.TTransportException
> >>         at
> >> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(
> RemoteInterpreter.java:373)
> >>         at
> >> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(
> LazyOpenInterpreter.java:111)
> >>         at
> >> org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
> >>         at
> >> org.apache.zeppelin.scheduler.JobProgressPoller.run(
> JobProgressPoller.java:51)
> >> Caused by: org.apache.thrift.transport.TTransportException
> >>         at
> >> org.apache.thrift.transport.TIOStreamTransport.read(
> TIOStreamTransport.java:132)
> >>         at
> >> org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> >>         at
> >> org.apache.thrift.protocol.TBinaryProtocol.readAll(
> TBinaryProtocol.java:429)
> >>         at
> >> org.apache.thrift.protocol.TBinaryProtocol.readI32(
> TBinaryProtocol.java:318)
> >>         at
> >> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
> TBinaryProtocol.java:219)
> >>         at
> >> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> >>         at
> >> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
> Client.recv_getProgress(RemoteInterpreterService.java:296)
> >>         at
> >> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
> Client.getProgress(RemoteInterpreterService.java:281)
> >>         at
> >> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(
> RemoteInterpreter.java:370)
> >>         ... 3 more
> >>
> >> Flink in local mode works fine on zeppelin.
> >> Could somebody point me to what i'm doing wrong ?
> >>
> >> Thanks a lot!
> >> Frank
> >>
> >>
> >>
> >
>

Mime
View raw message