flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Alber <alber.maximil...@gmail.com>
Subject Understanding the behavior
Date Thu, 11 Dec 2014 11:15:07 GMT
Hi Flinksters,

after mapping a data set, the only value seems to disappear. I cannot
explain this behavior. Maybe someone can help me?

In this code I have 4 versions, the first two do basically nothing, but
ensure us that there is actually a value inside the dataset.
Version 2 maps the vector to a new vector. But the result set is empty.
Version 3 the same.

What I would like to achieve is version 3 aka change the id value of the
vector. But somehow the vector disappears and the result is always an empty

val startWidth =
env.fromCollection[Vector](Seq(Vector.ones(config.dimensions) *
config.startWidth)) map {x => new Vector(0, x.values)}
val startUpdate =
env.fromCollection[Vector](Seq(Vector.ones(config.dimensions) * 0.01F)) map
{x => new Vector(1, x.values)}
val startLastGradient =
env.fromCollection[Vector](Seq(Vector.zeros(config.dimensions))) map {x =>
new Vector(2, x.values)}

var stepSet = startWidth union startUpdate union startLastGradient
stepSet = stepSet.iterate(1){
    stepSet =>
    // version 1
    val width = stepSet filter {_.id == 0};// works
    // version 2
    val width = stepSet filter {_.id == 0} map {x => x};// works
    // version 3
    val width = stepSet filter {_.id == 0} map {x => new Vector(-1,
x.values)};// does not work
    // version 4
    val width = stepSet filter {_.id == 0} map {x: Vector => new Vector(23,
Array(1.0F, 2.0F))};// does not work

I append you jar, source code and input files.
The program writes into the the out_file the width dataset.
You can change the code "versions" at line 353 cont.

May call the program with (you need to update jar, in_file, random_file,
set out_file as you want):
flink run the_jar_file '-c', 'bumpboost.Job', 'in_file=/tmp/tmpdW3O98',
'out_file=/tmp/tmp2RISRF', 'random_file=/tmp/tmpEN9XU7', 'dimensions=1',
'N=100', 'iterations=30', 'multi_bump_boost=1',
'gradient_descent_iterations=30', 'cache=False', 'start_width=1.0',
'min_width=-4', 'max_width=6', 'min_width_update=1e-08',

Thank you!

View raw message