flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juan Fumero <juan.jose.fumero.alfo...@oracle.com>
Subject Re: Deterministic map?
Date Tue, 14 Jul 2015 17:40:58 GMT
Hi Chiwan, 
  great thanks. Is there any example available? 

Regards
Juan

On Wed, 2015-07-15 at 01:19 +0900, Chiwan Park wrote:
> Hi, If you use `partitionCustom()` method [1] with custom partitioner, you can guarantee
the order of partition.
> 
> Regards,
> Chiwan Park
> 
> [1] https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/api/java/DataSet.html#partitionCustom(org.apache.flink.api.common.functions.Partitioner,%20int)
> 
> > On Jul 15, 2015, at 1:10 AM, Juan Fumero <juan.jose.fumero.alfonso@oracle.com>
wrote:
> > 
> > Hi, 
> >  I am doing pure map computation with typical benchmarks like
> > BlackScholes and NBody. 
> > 
> > I am using local configuration with multiple threads. It seems like,
> > inside the chuck (total size / numThreads) the order is correct. But the
> > ordering between chunks is not correct, giving an incorrect result at
> > the end.
> > 
> > What I mean by the order is, the correct result in the correct position
> > of the array. 
> > 
> > Is there any way to guarantee the result? 
> > 
> > Many thanks
> > Juan
> > 
> 
> 
> 
> 



Mime
View raw message