flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fridtjof Sander <fsan...@mailbox.tu-berlin.de>
Subject How to register aggregation convergence criterion to bulk iteration in scala API?
Date Thu, 28 Jan 2016 22:05:45 GMT
Hi,

I want to register a custom aggregation convergence criterion to a bulk 
iteration and I want to use the scala API.
It appears to me that this is not possible at the moment, right?

The AggregatorRegistry is exposed by IterativeDataSet.java, which is 
hidden by DataSet.scala:

   def iterate(maxIterations: Int)(stepFunction: (DataSet[T]) => 
DataSet[T]): DataSet[T] = {
     val iterativeSet =
       new IterativeDataSet[T](
         javaSet.getExecutionEnvironment,
         javaSet.getType,
         javaSet,
         maxIterations)

     val resultSet = stepFunction(wrap(iterativeSet))
     val result = iterativeSet.closeWith(resultSet.javaSet)
     wrap(result)
   }

I am aware of the iterateWithTermination-possibility and it's a 
work-around for me, but I guess the aggregated convergence criterion 
would be more efficient.
Am I missing something?

Best,
Fridtjof

Mime
View raw message