flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: flink dataStream operate dataSet
Date Mon, 05 Sep 2016 09:21:23 GMT
Hi,
right now it is not possible to mix the DataSet and the DataStream API. The
reason for the "task not serializable" error is that putting the DataSet
into the map function tries to serialize the DataSet, which is not possible.

Cheers,
Aljoscha

On Tue, 30 Aug 2016 at 16:31 <rimin515@sina.cn> wrote:

> Hi,
>      i have a problem,a dataStream read from rabbitMQ,and others data from
> a hbase table,which is a dataSet.Those two data from follow:
>
>      val words=connectHelper.readFromRabbitMq(...)  // words is
> DataStream[String]
>      val dataSet=HBaseWrite.fullScan(....)  //dataSet is
> DataSet[(int,String)]
>
>      words.map{ word =>
>          val res = dataSet.map{ y =>
>                        val score = computerScore(x,y)
>                    (word,score)
>               }
>          HBaseWrite.writeToTable(res,...,)
>      }
>
>    the  error is task not serializable,what is the solution?
>   under a DataStream, how to operate a DataSet?
>
>
>
>

Mime
View raw message