flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Serialisation problem
Date Sat, 12 Dec 2015 19:43:14 GMT
Hi,

Can you check the log output in your IDE or the log files of the Flink
client (./bin/flink). The TypeExtractor is logging why a POJO is not
recognized as a POJO.

The log statements look like this:

20:42:43,465 INFO  org.apache.flink.api.java.typeutils.TypeExtractor
      - class com.dataartisans.debug.MyPojo must have a default constructor
to be used as a POJO.



On Thu, Dec 10, 2015 at 11:24 PM, Abdulrahman kaitoua <
abdulrahman.kaitoua@outlook.com> wrote:

>
>
> Hello,
>
> I would like to hive directions to make my code work again (thanks in
> advance). My code used to work on versions equal or less than 9.1 but when
> i included 10 or 10.1 i got the following exception.
>
> This type
> (ObjectArrayTypeInfo<GenericType<it.polimi.genomics.core.DataTypes.GValue>>)
> cannot be used as key
>
> I understood that it is related to the serialisation of objects. I tried
> to follow the POJO building directions in
> https://cwiki.apache.org/confluence/display/FLINK/Type+System,+Type+Extraction,+Serialization
> with no luck to make it work.
>
> my dataset contains a set of tuples as key and one array of GValues, this
> is a snapshot of the GValue class.
>
>
> sealed trait GValue extends Serializable with Ordered[GValue]{
>   def compare(o : GValue) : Int = {
>     o match {
>       case GDouble(v) => this.asInstanceOf[GDouble].v compare v
>       case GString(v) => this.asInstanceOf[GString].v compare v
>       case GInt(v) => this.asInstanceOf[GInt].v compare v
>       case GNull() => 0
>     }
>   }
>   def equal(o : GValue) : Boolean = {
>     o match {
>       case GInt(value) => try{value.equals(o.asInstanceOf[GInt].v)} catch { case e
: Throwable => false }
>       case GDouble(value) => try{value.equals(o.asInstanceOf[GDouble].v)} catch {
case e : Throwable => false }
>       case GString(value) => try{value.equals(o.asInstanceOf[GString].v)} catch {
case e : Throwable => false }
>       case GNull() => o.isInstanceOf[GNull]
>       case _ => false
>     }
>   }
> }
>
> /**
>  * Represents a @GValue that contains an integer
>  * @deprecated
>  * @param v
>  */
> case class GInt(v: Int) extends GValue{
>   def this() = this(0)
>   override def toString() : String = {
>     v.toString
>   }
>   override def equals(other : Any) : Boolean = {
>     other match {
>       case GInt(value) => value.equals(v)
>       case _ => false
>     }
>   }
> }
>
> /**
>  * Represents a @GValue that contains a number as a @Double
>  * @param v number
>  */
> case class GDouble(v: Double) extends GValue {//with Ordered[GDouble]{
>
>   def this() = this(0.0)
>
>   override def equals(other : Any) : Boolean = {
>     other match {
>       case GDouble(value) => value.equals(v)
>       case _ => false
>     }
>   }
> }
>
> /**
>  * Represents a @GValue that contains a @String
>  * @param v string
>  */
> case class GString(v: String) extends GValue{
>   def this() = this(".")
>   override def toString() : String = {
>     v.toString
>   }
>   override def equals(other : Any) : Boolean = {
>     other match {
>       case GString(value) => value.equals(v)
>       case _ => false
>     }
>   }
> }
>
>
> Regards,
>
>
>
>
> *-----------------------------------------------------------------Abdulrahman
> Kaitoua-----------------------------------------------------------------Ph.D.
> Candidate at Politecnico Di Milano*
>
>

Mime
View raw message