flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sachin Goel <sachingoel0...@gmail.com>
Subject Re: Multiple restarts of Local Cluster
Date Wed, 02 Sep 2015 16:10:31 GMT
Okay. No problem.
Any suggestions for the correct context though? :')

I don't think something like a {{FlinkProgram}} class is a good idea [User
would need to override a {{program}} method and we will make sure the
cluster is setup only once and torn down properly only after the user code
finishes completely].

However, if this isn't so, the shutting down of cluster becomes impossible.
Can I assume however, that the actor system will be shut down automatically
when the main method exits? After all the JVM will terminate. If so, I can
make some changes in LocalExecutor to start up the cluster only once.

Regards
Sachin

-- Sachin Goel
Computer Science, IIT Delhi
m. +91-9871457685

On Wed, Sep 2, 2015 at 9:33 PM, Till Rohrmann <trohrmann@apache.org> wrote:

> Oh sorry, then I got the wrong context. I somehow thought it was about
> test cases because I read `MultipleProgramTestBase` etc. Sorry my bad.
>
> On Wed, Sep 2, 2015 at 6:00 PM, Sachin Goel <sachingoel0101@gmail.com>
> wrote:
>
>> I was under the impression that the @AfterClass annotation can only be
>> used in test classes.
>> Even so, the idea is that a user program running in the IDE should not be
>> starting up the cluster several times [my primary concern is the addition
>> of the persist operator], and we certainly cannot ask the user to terminate
>> the cluster after execution, while in local mode.
>>
>> -- Sachin Goel
>> Computer Science, IIT Delhi
>> m. +91-9871457685
>>
>> On Wed, Sep 2, 2015 at 9:19 PM, Till Rohrmann <trohrmann@apache.org>
>> wrote:
>>
>>> Why is it not possible to shut down the local cluster? Can’t you shut it
>>> down in the @AfterClass method?
>>> ​
>>>
>>> On Wed, Sep 2, 2015 at 4:56 PM, Sachin Goel <sachingoel0101@gmail.com>
>>> wrote:
>>>
>>>> Yes. That will work too. However, then it isn't possible to shut down
>>>> the local cluster. [Is it necessary to do so or does it shut down
>>>> automatically when the program exists? I'm not entirely sure.]
>>>>
>>>> -- Sachin Goel
>>>> Computer Science, IIT Delhi
>>>> m. +91-9871457685
>>>>
>>>> On Wed, Sep 2, 2015 at 7:59 PM, Stephan Ewen <sewen@apache.org> wrote:
>>>>
>>>>> Have a look at some other tests, like the checkpointing tests. They
>>>>> start one cluster manually and keep it running. They connect against
it
>>>>> using the remote environment ("localhost",
>>>>> miniCluster.getJobManagerRpcPort()).
>>>>>
>>>>> That works nicely...
>>>>>
>>>>> On Wed, Sep 2, 2015 at 4:23 PM, Sachin Goel <sachingoel0101@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all
>>>>>> While using LocalEnvironment, in case the program triggers execution
>>>>>> several times, the {{LocalFlinkMiniCluster}} is started as many times.
This
>>>>>> can consume a lot of time in setting up and tearing down the cluster.
>>>>>> Further, this hinders with a new functionality I'm working on based
on
>>>>>> persisted results.
>>>>>> One potential solution could be to follow the methodology in
>>>>>> `MultipleProgramsTestBase`. The user code then would have to reside
in a
>>>>>> fixed function name, instead of the main method. Or is that too cumbersome?
>>>>>>
>>>>>> Regards
>>>>>> Sachin
>>>>>> -- Sachin Goel
>>>>>> Computer Science, IIT Delhi
>>>>>> m. +91-9871457685
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message