flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Alexandrov <alexander.s.alexand...@gmail.com>
Subject Re: Should collect() and count() be treated as data sinks?
Date Thu, 02 Apr 2015 16:59:33 GMT
I have a similar issue here:

I would like to run a dataflow up to a particular point and materialize (in
memory) the intermediate result. Is this possible at the moment?

Regards,
Alex

2015-04-02 17:33 GMT+02:00 Felix Neutatz <neutatz@googlemail.com>:

> Hi,
>
> I have run the following program:
>
> final ExecutionEnvironment env =
> ExecutionEnvironment.getExecutionEnvironment();
>
> List l = Arrays.asList(new Tuple1<Long>(1L));
> TypeInformation t = TypeInfoParser.parse("Tuple1<Long>");
> DataSet<Tuple1<Long>> data = env.fromCollection(l, t);
>
> long value = data.count();
> System.out.println(value);
>
> env.execute("example");
>
>
> Since there is no "real" data sink, I get the following:
> Exception in thread "main" java.lang.RuntimeException: No data sinks have
> been created yet. A program needs at least one sink that consumes data.
> Examples are writing the data set or printing it.
>
> In my opinion, we should handle count() and collect() like print().
>
> What do you think?
>
> Best regards,
>
> Felix
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message