flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mxm <...@git.apache.org>
Subject [GitHub] flink pull request: implement a simple session management
Date Wed, 20 May 2015 10:09:41 GMT
Github user mxm commented on the pull request:

    Thanks for your feedback.
    >You introduce a separate session ID here. Why not use the JobID as the session ID?
This is how the logic in the JobManager (that attaches the graphs) currently works. Is there
any limitation with that approach that requires to introduce the new ID?
    Having two ids, `JobID` and the `SessionID`, gives us the possibility to submit multiple
independent job graphs in one session. Maybe that's a bit of an overkill since we can already
attach multiple independent branches to one job graph. 
    >You schedule the removeJob() call after job is done. Does that mean the job will be
removed after that time, even if new activity on that job happens?
    Currently, that is the case but this will change once we support to resume or extend jobs.
In this case, we can cancel the timer that removes the job.
    >The Client is bound to a session now. Should we tie it strictly to a session, meaning
it has to be initialized with a JobID, or lazily pick up the session. What happens if you
submit a JobGraph that comes from a different session? Does it track multiple sessions then?
    The Client is bound to a session if the `sessionID` is set, otherwise it defaults to null
and the job manager removes the job immediately after it has finished. I guess this is the
kind of lazy pick up you mentioned. It has the advantage that the session management is optional.
The job manager tracks multiple sessions. For each session it currently accepts multiple job
graphs. Like you suggested that might be unnecessary and we can just move to one job graph
per session.
    >The LocalExecutor used in the LocalExecutionEnvironment should probably allow you
to maintain sessions as well. It would need to be started on the first execution and stopped
when the environment terminates the session or runs out of scope.
    Yes, I didn't add anything there but you're right that it is a valid use even for local
    >I think it makes a lot of sense to automatically terminate sessions as soon as the
environment or the executor runs out of scope (becomes finalizable or garbage collectible).
    We can probably implement this by overriding the `finalize()` method of the `ExecutionEnvironment`

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message