flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Rohrmann (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5221) Checkpointed workless in Window Operator
Date Tue, 06 Dec 2016 12:51:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15725405#comment-15725405

Till Rohrmann commented on FLINK-5221:

Hi [~shijinkui], the Flink distribution also contains the Scala classes and the user function
is loaded with the user code class loader. Thus, you should not have the problem that you
encounter a class not found exception. Otherwise this points towards a bug in the system.

I think in general the decision to implement the Scala API on top of the Java API as a thin
layer is not a bad idea. This reduces complexity and helps to avoid code duplication. Furthermore,
I think that doing it vice versa would not be optimal. The reason is that Scala contains concepts,
such as traits, which you cannot use from Java. So for example, if you define a trait in Scala
(containing a method definition), then you cannot simply extend the trait and inherit this

Therefore, as far as I know, there are no plans to change this design fundamentally in the
near future.

> Checkpointed workless in Window Operator
> ----------------------------------------
>                 Key: FLINK-5221
>                 URL: https://issues.apache.org/jira/browse/FLINK-5221
>             Project: Flink
>          Issue Type: Bug
>          Components: DataStream API
>    Affects Versions: 1.2.0, 1.1.3
>         Environment: SUSE
>            Reporter: Syinchwun Leo
>              Labels: windows
>             Fix For: 1.2.0
>   Original Estimate: 12h
>  Remaining Estimate: 12h
> When window OPERATOR making checkpoint like this:
> class WindowStatistic extends WindowFunction[Event, Int, Tuple, TimeWindow] with Checkpointed[Option[List[Event]]]
>     override def appley().... 
>     override def snapshotState()...
>     override def restoreState()
> }
> Window Operator couldn't invoke user defined function "snapshotState()". In debug model,
line 123 in AbstractUdfStreamOperator.java returns false and can't make user defined state
when checking the window whether is a Checkpointed instance. I think there is something wrong
in userFunction var, it's a ScalaWindowFunctionWrapper object and it couldn't reflect if the
user defined window extend Checkpointed Interface. Actually, the user defined window is kept
in "func" var of userFunction. 

This message was sent by Atlassian JIRA

View raw message