flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9138) Enhance BucketingSink to also flush data by time interval
Date Wed, 09 May 2018 17:38:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16469170#comment-16469170

ASF GitHub Bot commented on FLINK-9138:

Github user glaksh100 commented on the issue:

    @fhueske  I gave it some thought and your suggestion makes sense to me. I have extended
`checkForInactiveBuckets` to include the rollover check. I have also updated Javadocs in a
few places:
    - Added a note in the top-level Javadocs to update functionality of `checkForInactiveBuckets()`
    - Updated JavaDocs for both `setBatchRolloverInterval()` and `setInactiveBucketThreshold()`
    - Updated JavaDoc for `checkForInactiveBuckets()`
    Let me know if the updates make sense and thank you for reviewing!

> Enhance BucketingSink to also flush data by time interval
> ---------------------------------------------------------
>                 Key: FLINK-9138
>                 URL: https://issues.apache.org/jira/browse/FLINK-9138
>             Project: Flink
>          Issue Type: Improvement
>          Components: filesystem-connector
>    Affects Versions: 1.4.2
>            Reporter: Narayanan Arunachalam
>            Priority: Major
> BucketingSink now supports flushing data to the file system by size limit and by period
of inactivity. It will be useful to also flush data by a specified time period. This way,
the data will be written out when write throughput is low but there is no significant time
period gaps between the writes. This reduces ETA for the data in the file system and should
help move the checkpoints faster as well.

This message was sent by Atlassian JIRA

View raw message