flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2834) Global round-robin for temporary directories
Date Tue, 20 Oct 2015 17:13:27 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14965408#comment-14965408
] 

ASF GitHub Bot commented on FLINK-2834:
---------------------------------------

Github user StephanEwen commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1272#discussion_r42524229
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/disk/iomanager/FileIOChannel.java
---
    @@ -154,20 +155,25 @@ private static String randomString(Random random) {
     	public static final class Enumerator {
     
     		private final File[] paths;
    -		
    +
     		private final String namePrefix;
     
    -		private int counter;
    +		private int localCounter;
    +
    +		private static AtomicInteger globalCounter = new AtomicInteger();
    --- End diff --
    
    Minor (optional) comment: In the (undocumented) code style that Flink follows implicitly,
the static member fields are the first in the class, before the instance member fields.


> Global round-robin for temporary directories
> --------------------------------------------
>
>                 Key: FLINK-2834
>                 URL: https://issues.apache.org/jira/browse/FLINK-2834
>             Project: Flink
>          Issue Type: Improvement
>          Components: Distributed Runtime
>    Affects Versions: 0.10
>            Reporter: Greg Hogan
>            Assignee: Greg Hogan
>            Priority: Minor
>
> Currently each {{FileIOChannel}} performs a local round-robin among temporary directories.
Initially every writer will be concurrently spilling to the same first directory, then the
same second directory, etc. Even when evenly dispersed this effect will naturally occur due
to the bunching effect.
> One solution is to use a global counter when selecting the temporary path.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message