flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Alexandrov <alexander.s.alexand...@gmail.com>
Subject Re: DeltaIterations: shrink solution set
Date Tue, 10 Feb 2015 17:54:14 GMT
I am not sure whether this is supported at the moment. The only workaround
I could think of is indeed to use a boolean flag that indicates whether the
element has been deleted or not.

An alternative approach is to ditch Flink's native iteration construct and
write your intermediate results to Tachyon or HDFS after each iteration
using the TypeInfoInput/OutputFormats. You then have full control how the
old and the new solutions sets should be merged.

BTW can you share some details about that particular algorithm? I was
thinking about examples iterative algorithms with this property...


2015-02-10 14:18 GMT+01:00 Kruse, Sebastian <Sebastian.Kruse@hpi.de>:

>  Hi everyone,
> From playing around a bit around with delta iterations, I saw that you can
> update elements from the solution set and add new elements. My question is:
> is it possible to remove elements from the solution set (apart from marking
> them as “deleted” somehow)?
> My use case at hand for this is the following: In each iteration, I
> generate candidate solutions that I want to verify within the next
> iteration. If verification fails, I would like to remove them from the
> solution set, otherwise retain them.
> Thanks,
> Sebastian

View raw message