flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rtudoran <...@git.apache.org>
Subject [GitHub] flink issue #3590: [FLINK-5654] - Add processing time OVER RANGE BETWEEN x P...
Date Thu, 23 Mar 2017 08:40:37 GMT
Github user rtudoran commented on the issue:

    @fhueske @sunjincheng121 
    I have done another commit (actually i did it last evening but the network of the machine
was down and did not push it). What i did was:
    1)address the formatting issues (i hope i did not miss (too) many)
    2) added some new tests including for processingtime. I did not find the reference to
what @fhueske mentioned but i created a custom source that emits events 1 second apart. This
gives us a good framework about events arriving with processing time 1 second apart, so we
can run any tests and validate. I think this is quite relevant
    3) I did update the processing functions. However, i am still using the ValueState[Queue[JTuple2[Long,Row]]]=
_ for buffering the events. As per my previosu mentioning - i strongly believe this is a better
approach. Having the events sorted by their order is an advantage that we should not loose.
Otherwise we will pay a high price later (sorting, limit, top, distinct...). We pay some price
for serialziation, but as we do this over a full object it should be relatively ok rather
than serializing independent objects (like in hash map...IMHO)
    4) I kept the partitioned/non partitioned cases separately ... as per initial argument
of Fabian (i think it is worth paying the price of having an extra class) not to add extra
operators that need to be maintained, dedicate resource, monitored for liveness and redeploy
in case of failures.

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message