flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Join two Datasets --> InvalidProgramException
Date Tue, 09 Feb 2016 13:28:13 GMT
String is perfectly fine as key.
Looks like SourceA / SourceB are not correctly identified as Pojos.

2016-02-09 14:25 GMT+01:00 Dominique Rondé <dominique.ronde@codecentric.de>:

> Sorry, i was out for lunch. Maybe the problem is that sessionID is a
> String?
>
> public abstract class Parent{
>   private Date eventDate;
>   private EventType eventType;
>   private String sessionId;
>
> public Parent() { }
> //GETTER & SETTER
> }
>
> public class SourceA extends Parent{
>   private Boolean outboundMessage;
>   private String soapMessage;
>
> public SourceA () {
>     super();
>  }
> //GETTER & SETTER
> }
>
> public class SourceB extends Parent{
>   private Integer id;
>   private String username;
>
> public SourceB () {
>     super();
>  }
> //GETTER & SETTER
>
> }
>
> Am 09.02.2016 um 12:06 schrieb Till Rohrmann:
>
> Could you share the code for your types SourceA and SourceB. It seems as
> if Flink does not recognize them to be POJOs because he assigned them the
> GenericType type. Either there is something wrong with the type extractor
> or your implementation does not fulfil the requirements for POJOs, as
> indicated by Chiwan.
>
> Cheers,
> Till
> ​
>
> On Tue, Feb 9, 2016 at 11:53 AM, Dominique Rondé <
> dominique.ronde@codecentric.de> wrote:
>
>> The fields in SourceA and SourceB are private but have public getters and
>> setters. The classes provide an empty and public constructor.
>> Am 09.02.2016 11:47 schrieb "Chiwan Park" <chiwanpark@apache.org>:
>>
>>> Oh, the fields in SourceA have public getters. Does the fields in
>>> SourceA have public setter? SourceA needs public setter for private fields.
>>>
>>> Regards,
>>> Chiwan Park
>>>
>>> > On Feb 9, 2016, at 7:45 PM, Chiwan Park < <chiwanpark@apache.org>
>>> chiwanpark@apache.org> wrote:
>>> >
>>> > Hi Dominique,
>>> >
>>> > It seems that `SourceA` is not dealt as POJO. Are all fields in
>>> SourceA public? There are some requirements for POJO classes [1].
>>> >
>>> > [1]:
>>> https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/programming_guide.html#pojos
>>> >
>>> > Regards,
>>> > Chiwan Park
>>> >
>>> >> On Feb 9, 2016, at 7:42 PM, Dominique Rondé <
>>> dominique.ronde@codecentric.de> wrote:
>>> >>
>>> >> Hi  folks,
>>> >>
>>> >> i try to join two datasets containing some PoJos. Each PoJo inherit
a
>>> field "sessionId" from the parent class. The field is private but has a
>>> public getter.
>>> >>
>>> >> The join is like this:
>>> >> DataSet<Tuple2<SourceA,SourceB>> joinedDataSet =
>>> sourceA.join(SourceB).where("sessionId").equalTo("sessionId");
>>> >>
>>> >> But the result is the following execption:
>>> >>
>>> >> Exception in thread "main"
>>> org.apache.flink.api.common.InvalidProgramException: This type
>>> (GenericType<x.y.z.service.eventstore.dto.SourceA>) cannot be used as key.
>>> >>    at
>>> org.apache.flink.api.java.operators.Keys$ExpressionKeys.<init>(Keys.java:287)
>>> >>    at
>>> org.apache.flink.api.java.operators.JoinOperator$JoinOperatorSets.where(JoinOperator.java:890)
>>> >>    at
>>> x.y.z.eventstore.processing.pmc.PmcProcessor.main(PmcProcessor.java:55)
>>> >>
>>> >> I spend some time with google around but I don't get an idea what is
>>> wrong. I hope some of you can give me a hint...
>>> >>
>>> >> Greets
>>> >> Dominique
>>> >>
>>> >
>>>
>>>
>
> --
> Dominique Rondé | Senior Consultant
>
> codecentric AG | Kreuznacherstrasse 30 | 60486 Frankfurt | Deutschland
> mobil: +49 (0) 172.7182592www.codecentric.de | blog.codecentric.de | www.meettheexperts.de
| www.more4fi.de
>
> Sitz der Gesellschaft: Solingen | HRB 25917| Amtsgericht Wuppertal
> Vorstand: Michael Hochgürtel . Mirko Novakovic . Rainer Vehns
> Aufsichtsrat: Patric Fedlmeier (Vorsitzender) . Klaus Jäger . Jürgen Schütz
>
>

Mime
View raw message