hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9782) RollingFileSystemSink should have configurable roll interval
Date Fri, 01 Apr 2016 16:48:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15221953#comment-15221953
] 

Daniel Templeton commented on HDFS-9782:
----------------------------------------

The granularity of the directory names is 1 minute, but that doesn't require that the interval
have a 1-minute granularity.  I was thinking of the case of a 90-second interval.  If you
think that has the potential to create confusion, I can drop seconds from the allowed units.

> RollingFileSystemSink should have configurable roll interval
> ------------------------------------------------------------
>
>                 Key: HDFS-9782
>                 URL: https://issues.apache.org/jira/browse/HDFS-9782
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
>         Attachments: HDFS-9782.001.patch, HDFS-9782.002.patch, HDFS-9782.003.patch, HDFS-9782.004.patch,
HDFS-9782.005.patch, HDFS-9782.006.patch
>
>
> Right now it defaults to rolling at the top of every hour.  Instead that interval should
be configurable.  The interval should also allow for some play so that all hosts don't try
to flush their files simultaneously.
> I'm filing this in HDFS because I suspect it will involve touching the HDFS tests.  If
it turns out not to, I'll move it into common instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message