flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2458) Distributed Cache doesn't work with iterations.
Date Fri, 07 Aug 2015 12:23:46 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661751#comment-14661751
] 

ASF GitHub Bot commented on FLINK-2458:
---------------------------------------

Github user sachingoel0101 commented on a diff in the pull request:

    https://github.com/apache/flink/pull/970#discussion_r36513272
  
    --- Diff: flink-core/src/main/java/org/apache/flink/api/common/operators/CollectionExecutor.java
---
    @@ -501,4 +536,22 @@ public int getSuperstepNumber() {
     			return (T) previousAggregates.get(name);
     		}
     	}
    +
    +	private static final class DoingNothing implements Callable<Path>{
    +		private Path entry;
    +
    +		public DoingNothing(Path entry){
    +			this.entry = entry;
    +		}
    +
    +		@Override
    +		public Path call() throws IOException{
    +			try{
    +				LocalFileSystem fs = (LocalFileSystem) entry.getFileSystem();
    +				return entry.isAbsolute() ? new Path(entry.toUri().getPath()): new Path(fs.getWorkingDirectory(),entry);
    +			} catch (ClassCastException e){
    +				throw new RuntimeException("Collection execution must have only local file paths");
    --- End diff --
    
    Yeah. That would make more sense.


> Distributed Cache doesn't work with iterations.
> -----------------------------------------------
>
>                 Key: FLINK-2458
>                 URL: https://issues.apache.org/jira/browse/FLINK-2458
>             Project: Flink
>          Issue Type: Bug
>            Reporter: Sachin Goel
>            Assignee: Sachin Goel
>
> Distributed cache files cannot be accessed when working in Iterative contexts. While
constructing the context in {{AbstractIterativePactTask}}, the distributed entries are not
passed on to runtime context.
> My commit will add this and also unify the distributed cache system to work everywhere,
including in the Collection Environment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message