apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2096) Add blockThreshold parameter to FSInputModule
Date Thu, 19 May 2016 02:55:13 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290349#comment-15290349
] 

ASF GitHub Bot commented on APEXMALHAR-2096:
--------------------------------------------

Github user gauravgopi123 commented on a diff in the pull request:

    https://github.com/apache/incubator-apex-malhar/pull/283#discussion_r63816277
  
    --- Diff: library/src/main/java/com/datatorrent/lib/io/fs/FSInputModule.java ---
    @@ -57,6 +58,8 @@
       private long blockSize;
       private boolean sequencialFileRead = false;
       private int readersCount;
    +  @Min(1)
    +  protected int blocksThreshold;
    --- End diff --
    
    why not initialize it to 1 so that user don't have to set anything if user goes with default
value


> Add blockThreshold parameter to FSInputModule
> ---------------------------------------------
>
>                 Key: APEXMALHAR-2096
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2096
>             Project: Apache Apex Malhar
>          Issue Type: Improvement
>            Reporter: Priyanka Gugale
>            Assignee: Priyanka Gugale
>
> FileSplitter is very fast, the downstream operators can't work at that speed, so we need
to limit the speed with which fileSplitter works.
> One way to do is limit number of blocks emitted per window.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message