flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Understanding the behavior
Date Thu, 11 Dec 2014 11:41:05 GMT
Hi,
the reasons is that you filter out the items. In your first iteration,
a new element is created that has a -1 and a 23 as the first field,
respectively for version 3 and version 4. In the second iteration, you
filter out all elements that do not have a "0" as the first field.
Thus you arrive at an empty set.

Cheers,
Aljoscha

On Thu, Dec 11, 2014 at 12:15 PM, Maximilian Alber
<alber.maximilian@gmail.com> wrote:
> Hi Flinksters,
>
> after mapping a data set, the only value seems to disappear. I cannot
> explain this behavior. Maybe someone can help me?
>
> In this code I have 4 versions, the first two do basically nothing, but
> ensure us that there is actually a value inside the dataset.
> Version 2 maps the vector to a new vector. But the result set is empty.
> Version 3 the same.
>
> What I would like to achieve is version 3 aka change the id value of the
> vector. But somehow the vector disappears and the result is always an empty
> set.
>
> val startWidth =
> env.fromCollection[Vector](Seq(Vector.ones(config.dimensions) *
> config.startWidth)) map {x => new Vector(0, x.values)}
> val startUpdate =
> env.fromCollection[Vector](Seq(Vector.ones(config.dimensions) * 0.01F)) map
> {x => new Vector(1, x.values)}
> val startLastGradient =
> env.fromCollection[Vector](Seq(Vector.zeros(config.dimensions))) map {x =>
> new Vector(2, x.values)}
>
> var stepSet = startWidth union startUpdate union startLastGradient
> stepSet = stepSet.iterate(1){
>     stepSet =>
>     // version 1
>     val width = stepSet filter {_.id == 0};// works
>     // version 2
>     val width = stepSet filter {_.id == 0} map {x => x};// works
>     // version 3
>     val width = stepSet filter {_.id == 0} map {x => new Vector(-1,
> x.values)};// does not work
>     // version 4
>     val width = stepSet filter {_.id == 0} map {x: Vector => new Vector(23,
> Array(1.0F, 2.0F))};// does not work
>   width
> }
>
>
> I append you jar, source code and input files.
> The program writes into the the out_file the width dataset.
> You can change the code "versions" at line 353 cont.
>
> May call the program with (you need to update jar, in_file, random_file, set
> out_file as you want):
> flink run the_jar_file '-c', 'bumpboost.Job', 'in_file=/tmp/tmpdW3O98',
> 'out_file=/tmp/tmp2RISRF', 'random_file=/tmp/tmpEN9XU7', 'dimensions=1',
> 'N=100', 'iterations=30', 'multi_bump_boost=1',
> 'gradient_descent_iterations=30', 'cache=False', 'start_width=1.0',
> 'min_width=-4', 'max_width=6', 'min_width_update=1e-08',
> 'max_width_update=10'
>
> Thank you!
> Cheers,
> Max

Mime
View raw message