flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9280) Extend JobSubmitHandler to accept jar files
Date Wed, 13 Jun 2018 12:43:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511059#comment-16511059

ASF GitHub Bot commented on FLINK-9280:

Github user zentol commented on a diff in the pull request:

    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/FileUploadHandler.java
    @@ -89,7 +89,7 @@ protected void channelRead0(final ChannelHandlerContext ctx, final HttpObject
     			final HttpContent httpContent = (HttpContent) msg;
    -			while (currentHttpPostRequestDecoder.hasNext()) {
    --- End diff --
    The failure was triggered every-time i attempted to upload anything, but i may very well
depend on the payload size.
    Your analysis is mostly correct, what's missing is that decoder switches into the `EPILOGUE`
state when being offered a `LastHttpContent`.
    If this last message is not empty the exception is not thrown since data is still available,
checked in `hasNext` via `this.bodyListHttpDataRank >= this.bodyListHttpData.size()`.

> Extend JobSubmitHandler to accept jar files
> -------------------------------------------
>                 Key: FLINK-9280
>                 URL: https://issues.apache.org/jira/browse/FLINK-9280
>             Project: Flink
>          Issue Type: New Feature
>          Components: Job-Submission, REST
>    Affects Versions: 1.5.0
>            Reporter: Chesnay Schepler
>            Assignee: Chesnay Schepler
>            Priority: Critical
>             Fix For: 1.6.0, 1.5.1
> The job submission through the CLI first uploads all require jars to the blob server,
sets the blob keys in the jobgraph, and then uploads this graph to The {{JobSubmitHandler}}
which submits it to the Dispatcher.
> This process has the downside that it requires jars to be uploaded to the blobserver
before submitting the job graph, which does not happen via REST.
> I propose an extension to the the {{JobSubmitHandler}} to also accept an optional list
of jar files, that were previously uploaded through the {{JarUploadHandler}}. If present,
the handler would upload these jars to the blobserver and set the blob keys.

This message was sent by Atlassian JIRA

View raw message