apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2004) TimeBasedDirectoryScanner keep reading same file
Date Mon, 29 Feb 2016 20:53:18 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172589#comment-15172589
] 

ASF GitHub Bot commented on APEXMALHAR-2004:
--------------------------------------------

Github user chandnisingh commented on a diff in the pull request:

    https://github.com/apache/incubator-apex-malhar/pull/203#discussion_r54475635
  
    --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/FileSplitterInput.java ---
    @@ -439,12 +439,12 @@ protected ScannedFileInfo createScannedFileInfo(Path parentPath,
FileStatus pare
           ScannedFileInfo info;
           if (rootPath == null) {
             info = parentStatus.isDirectory() ?
    -          new ScannedFileInfo(parentPath.toUri().getPath(), childPath.getName(), parentStatus.getModificationTime())
:
    -          new ScannedFileInfo(null, childPath.toUri().getPath(), parentStatus.getModificationTime());
    +          new ScannedFileInfo(parentPath.toUri().getPath(), childPath.getName(), childStatus.getModificationTime())
:
    +          new ScannedFileInfo(null, childPath.toUri().getPath(), childStatus.getModificationTime());
    --- End diff --
    
    please remove ```@SuppressWarnings("UnusedParameters")``` with ```childStatus``` in the
method definition


> TimeBasedDirectoryScanner keep reading same file
> ------------------------------------------------
>
>                 Key: APEXMALHAR-2004
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2004
>             Project: Apache Apex Malhar
>          Issue Type: Bug
>    Affects Versions: 3.3.0
>            Reporter: Tushar Gosavi
>            Assignee: Tushar Gosavi
>
> The recorded last modification time is of the parent directory, If modification time
> last recorded modification time then we will read the file again.
> In case where directory modification time < file modification time, we will keep on
reading the file again and again.
> Fix is to record file modification time instead of parent modification time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message