flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Serialisation problem
Date Mon, 14 Dec 2015 09:42:22 GMT
Hi,
the problem could be that GValue is not Comparable. Could you try making it extend Comparable
(The Java Comparable).

Cheers,
Aljoscha
> On 12 Dec 2015, at 20:43, Robert Metzger <rmetzger@apache.org> wrote:
> 
> Hi,
> 
> Can you check the log output in your IDE or the log files of the Flink client (./bin/flink).
The TypeExtractor is logging why a POJO is not recognized as a POJO.
> 
> The log statements look like this:
> 
> 20:42:43,465 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - class
com.dataartisans.debug.MyPojo must have a default constructor to be used as a POJO.
> 
> 
> 
> On Thu, Dec 10, 2015 at 11:24 PM, Abdulrahman kaitoua <abdulrahman.kaitoua@outlook.com>
wrote:
> 
> 
> Hello,
> 
> I would like to hive directions to make my code work again (thanks in advance). My code
used to work on versions equal or less than 9.1 but when i included 10 or 10.1 i got the following
exception.
> 
> This type (ObjectArrayTypeInfo<GenericType<it.polimi.genomics.core.DataTypes.GValue>>)
cannot be used as key
> 
> I understood that it is related to the serialisation of objects. I tried to follow the
POJO building directions in https://cwiki.apache.org/confluence/display/FLINK/Type+System,+Type+Extraction,+Serialization
> with no luck to make it work.
> 
> my dataset contains a set of tuples as key and one array of GValues, this is a snapshot
of the GValue class.
> 
> 
> sealed trait GValue extends Serializable with Ordered[GValue]{
>   def compare(o : GValue) : Int = {
>     o match {
>       case GDouble(v) => this.asInstanceOf[GDouble].v compare v
>       case GString(v) => this.asInstanceOf[GString].v compare v
>       case GInt(v) => this.asInstanceOf[GInt].v compare v
>       case GNull() => 0
>     }
>   }
>   def equal(o : GValue) : Boolean = {
>     o match {
>       case GInt(value) => try{value.equals(o.asInstanceOf[GInt].v)} catch { case e
: Throwable => false }
>       case GDouble(value) => try{value.equals(o.asInstanceOf[GDouble].v)} catch {
case e : Throwable => false }
>       case GString(value) => try{value.equals(o.asInstanceOf[GString].v)} catch {
case e : Throwable => false }
>       case GNull() => o.isInstanceOf[GNull]
>       case _ => false
>     }
>   }
> }
> 
> /**
>  * Represents a @GValue that contains an integer
>  * @deprecated
>  * @param v
>  */
> case class GInt(v: Int) extends GValue{
>   def this() = this(0)
>   override def toString() : String = {
>     v.toString
>   }
>   override def equals(other : Any) : Boolean = {
>     other match {
>       case GInt(value) => value.equals(v)
>       case _ => false
>     }
>   }
> }
> 
> /**
>  * Represents a @GValue that contains a number as a @Double
>  * @param v number
>  */
> case class GDouble(v: Double) extends GValue {//with Ordered[GDouble]{
> 
>   def this() = this(0.0)
> 
>   override def equals(other : Any) : Boolean = {
>     other match {
>       case GDouble(value) => value.equals(v)
>       case _ => false
>     }
>   }
> }
> 
> /**
>  * Represents a @GValue that contains a @String
>  * @param v string
>  */
> case class GString(v: String) extends GValue{
>   def this() = this(".")
>   override def toString() : String = {
>     v.toString
>   }
>   override def equals(other : Any) : Boolean = {
>     other match {
>       case GString(value) => value.equals(v)
>       case _ => false
>     }
>   }
> }
> 
> Regards,
> 
> -----------------------------------------------------------------
> Abdulrahman Kaitoua
> -----------------------------------------------------------------
> Ph.D. Candidate at Politecnico Di Milano
> 
> 


Mime
View raw message