flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Liu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (FLINK-5668) passing taskmanager configuration through taskManagerEnv instead of file
Date Thu, 13 Apr 2017 00:54:41 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15966934#comment-15966934
] 

Bill Liu edited comment on FLINK-5668 at 4/13/17 12:53 AM:
-----------------------------------------------------------

[~StephanEwen]  and [~wheat9]
Sorry to raise this issue again.
I agree that  Flink Runtime needs a distributed file system, but I don't get why it need to
be writable.

For the example of persistent {code} Jobgraph {code}, it will be even better if user has the
choice to give path of  {code} Jobgraph {code} directly.
Furthermore it's the Flink Client writing {code} Jobgraph {code} not the Flink Runtime.

For my case, I would like to have the {code} Jobgraph {code} and  jars in a http server, 
which are used as resources to bootstrap the Flink Job on Yarn.

So my point is the Flink Runtime shouldn't assume {code}yarn.deploy.fs{code} is writable.





was (Author: bill.liu8904):
[~StephanEwen]  and [~wheat9]
Sorry to raise this issue again.
I agree that  Flink Runtime needs a distributed file system, but I don't get why it need to
be writable.

For the example of persistent `Jobgraph`, it will be even better if user has the choice to
give path of  `JobGraph` directly.
Furthermore it's the Flink Client writing `Jobgraph` not the Flink Runtime.

For my case, I would like to have the `JobGraph` and  jars in a http server,  which are used
as resources to bootstrap the Flink Job on Yarn.

So my point is the Flink Runtime shouldn't assume `yarn.deploy.fs` is writable.




> passing taskmanager configuration through taskManagerEnv instead of file
> ------------------------------------------------------------------------
>
>                 Key: FLINK-5668
>                 URL: https://issues.apache.org/jira/browse/FLINK-5668
>             Project: Flink
>          Issue Type: Improvement
>          Components: YARN
>            Reporter: Bill Liu
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> When create a Flink cluster on Yarn,  JobManager depends on  HDFS to share  taskmanager-conf.yaml
 with TaskManager.
> It's better to share the taskmanager-conf.yaml  on JobManager Web server instead of HDFS,
which could reduce the HDFS dependency  at job startup.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message