flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leonard Wolters <leon...@sagent.io>
Subject Re: Zeppelin Integration
Date Wed, 04 Nov 2015 16:12:53 GMT
Indeed very nice! Thanks
On Nov 4, 2015 5:04 PM, "Till Rohrmann" <trohrmann@apache.org> wrote:

> Really cool tutorial Trevor :-)
>
> On Wed, Nov 4, 2015 at 3:26 PM, Robert Metzger <rmetzger@apache.org>
> wrote:
>
>> For those interested, Trevor wrote a blog post describing how to setup
>> Spark, Flink and Zeppelin, both locally and on clusters:
>> http://trevorgrant.org/2015/11/03/apache-casserole-a-delicious-big-data-recipe-for-the-whole-family/
>> Thanks Trevor for the great tutorial!
>>
>> On Thu, Oct 22, 2015 at 4:23 PM, Till Rohrmann <trohrmann@apache.org>
>> wrote:
>>
>>> Hi Trevor,
>>>
>>> that’s actually my bad since I only tested my branch against a remote
>>> cluster. I fixed the problem (not properly starting the
>>> LocalFlinkMiniCluster) so that you can now use Zeppelin also in local
>>> mode. Just check out my branch again.
>>>
>>> Cheers,
>>> Till
>>> ​
>>>
>>> On Wed, Oct 21, 2015 at 10:00 PM, Trevor Grant <trevor.d.grant@gmail.com
>>> > wrote:
>>>
>>>> Hey Till,
>>>>
>>>> I cloned your branch of Zeplin and while it will compile, it fails
>>>> tests on timeout, which consequently was the same issue I was having when
>>>> trying to use Zeppelin.
>>>>
>>>> Ideas?
>>>>
>>>>
>>>> -------------------------------------------------------------------------------
>>>> Test set: org.apache.zeppelin.flink.FlinkInterpreterTest
>>>>
>>>> -------------------------------------------------------------------------------
>>>> Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 100.347
>>>> sec <<< FAILURE! - in org.apache.zeppelin.flink.FlinkInterpreterTest
>>>> org.apache.zeppelin.flink.FlinkInterpreterTest  Time elapsed: 100.347
>>>> sec  <<< ERROR!
>>>> java.util.concurrent.TimeoutException: Futures timed out after [100000
>>>> milliseconds]
>>>> at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
>>>> at
>>>> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
>>>> at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
>>>> at
>>>> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
>>>> at scala.concurrent.Await$.result(package.scala:107)
>>>> at
>>>> org.apache.flink.runtime.minicluster.FlinkMiniCluster.getLeaderIndex(FlinkMiniCluster.scala:171)
>>>> at
>>>> org.apache.flink.runtime.minicluster.LocalFlinkMiniCluster.getLeaderRPCPort(LocalFlinkMiniCluster.scala:132)
>>>> at
>>>> org.apache.zeppelin.flink.FlinkInterpreter.getPort(FlinkInterpreter.java:136)
>>>> at
>>>> org.apache.zeppelin.flink.FlinkInterpreter.open(FlinkInterpreter.java:98)
>>>> at
>>>> org.apache.zeppelin.flink.FlinkInterpreterTest.setUp(FlinkInterpreterTest.java:42)
>>>>
>>>> org.apache.zeppelin.flink.FlinkInterpreterTest  Time elapsed: 100.347
>>>> sec  <<< ERROR!
>>>> java.lang.NullPointerException: null
>>>> at
>>>> org.apache.zeppelin.flink.FlinkInterpreter.close(FlinkInterpreter.java:221)
>>>> at
>>>> org.apache.zeppelin.flink.FlinkInterpreterTest.tearDown(FlinkInterpreterTest.java:48)
>>>>
>>>>
>>>>
>>>> Trevor Grant
>>>> Data Scientist
>>>> https://github.com/rawkintrevo
>>>> http://stackexchange.com/users/3002022/rawkintrevo
>>>>
>>>> *"Fortunate is he, who is able to know the causes of things."  -Virgil*
>>>>
>>>>
>>>> On Wed, Oct 21, 2015 at 11:57 AM, Till Rohrmann <trohrmann@apache.org>
>>>> wrote:
>>>>
>>>>> Hi Trevor,
>>>>>
>>>>> in order to use Zeppelin with a different Flink version in local mode,
>>>>> meaning that Zeppelin starts a LocalFlinkMiniCluster when executing
>>>>> your jobs, you have to build Zeppelin and change the flink.version
>>>>> property in the zeppelin/flink/pom.xml file to the version you want
>>>>> to use.
>>>>>
>>>>> If you want to let Zeppelin submit jobs to a remote cluster, you
>>>>> should build Zeppelin with the version of your cluster. That’s because
>>>>> internally Zeppelin will use this version to construct a JobGraph
>>>>> which is then submitted to the cluster. In order to configure the remote
>>>>> cluster, you have to go the *Interpreter* page and scroll down to the
>>>>> *flink* section. There you have to specify the address of your
>>>>> cluster under *host* and the port under *port*. This should then be
>>>>> used to submit jobs to the Flink cluster.
>>>>>
>>>>> I hope this answers your question.
>>>>>
>>>>> Btw: If you want to use Zeppelin with the latest Flink 0.10-SNAPSHOT
>>>>> version, you should checkout my branch
>>>>> https://github.com/tillrohrmann/incubator-zeppelin/tree/flink-0.10-SNAPSHOT
>>>>> where I’ve made the necessary changes.
>>>>>
>>>>> Cheers,
>>>>> Till
>>>>> ​
>>>>>
>>>>> On Wed, Oct 21, 2015 at 5:00 PM, Trevor Grant <
>>>>> trevor.d.grant@gmail.com> wrote:
>>>>>
>>>>>> I'm setting up some Flink/Spark/Zeppelin at work.  Spark+Zeppelin
>>>>>> seems to be relatively well supported and configurable but the Flink
is not
>>>>>> so much.
>>>>>>
>>>>>> I want Zeppelin to run against my 0.10 build instead of the 0.6 build
>>>>>> that ships with Zeppelin.  My best guess at the moment on how to
accomplish
>>>>>> this is to create a symbolic link from the /opt/zepplin/flink folder
to
>>>>>> /opt/flink-0.10, but this feels dirty and wrong.
>>>>>>
>>>>>> Does anyone out there have any experience connecting Zeppelin to
a
>>>>>> non-prepackaged Flink build?
>>>>>>
>>>>>> I feel like there is a great opporutnity for a HOWTO write up if
non
>>>>>> currently exists.
>>>>>>
>>>>>> I'm asking on the Zeppelin user mailing list too as soon as I am
>>>>>> added.
>>>>>>
>>>>>> Thanks for any help
>>>>>>
>>>>>> tg
>>>>>>
>>>>>>
>>>>>> Trevor Grant
>>>>>> Data Scientist
>>>>>> https://github.com/rawkintrevo
>>>>>> http://stackexchange.com/users/3002022/rawkintrevo
>>>>>>
>>>>>> *"Fortunate is he, who is able to know the causes of things."
>>>>>>  -Virgil*
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message