apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2069) FileSplitterInput and TimeBasedDirectoryScanner - move operational fields initialization from constructor to setup
Date Wed, 06 Jul 2016 15:27:11 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364461#comment-15364461
] 

ASF GitHub Bot commented on APEXMALHAR-2069:
--------------------------------------------

Github user vrozov commented on a diff in the pull request:

    https://github.com/apache/apex-malhar/pull/333#discussion_r69752755
  
    --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/FileSplitterInput.java ---
    @@ -80,29 +80,29 @@
     {
       @NotNull
       private WindowDataManager windowDataManager;
    -  @NotNull
    -  protected final transient LinkedList<ScannedFileInfo> currentWindowRecoveryState;
    +
    +  protected transient LinkedList<ScannedFileInfo> currentWindowRecoveryState;
     
       @Valid
       @NotNull
       private TimeBasedDirectoryScanner scanner;
    -  @NotNull
    +
       private Map<String, Map<String, Long>> referenceTimes;
     
       private transient long sleepMillis;
     
       public FileSplitterInput()
       {
         super();
    -    currentWindowRecoveryState = Lists.newLinkedList();
         windowDataManager = new WindowDataManager.NoopWindowDataManager();
    -    referenceTimes = Maps.newHashMap();
         scanner = new TimeBasedDirectoryScanner();
       }
     
       @Override
       public void setup(Context.OperatorContext context)
       {
    +    currentWindowRecoveryState = Lists.newLinkedList();
    +    referenceTimes = Maps.newHashMap();
    --- End diff --
    
    `referenceTimes` is not marked as transient, so it will be stored during checkpointing
and restored back from a checkpoint in case of failover. It is necessary to check referenceTimes
for `null`. 


> FileSplitterInput and TimeBasedDirectoryScanner - move operational fields initialization
from constructor to setup
> ------------------------------------------------------------------------------------------------------------------
>
>                 Key: APEXMALHAR-2069
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2069
>             Project: Apache Apex Malhar
>          Issue Type: Improvement
>            Reporter: Vlad Rozov
>
> For example, there is no need for scanService to be initialized in the constructor. It
should be done during operator setup().



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message