flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arvid Heise <arvid.he...@gmail.com>
Subject Re: ReduceGroup fails on server
Date Fri, 30 Jan 2015 17:04:13 GMT
Sorry building the 0.9 branch took longer than expected. I will follow that
up on Monday.

Alternatively I would be grateful for a 0.8.0 patch ;)

Best,

Arvid

On Fri, Jan 30, 2015 at 5:09 PM, Aljoscha Krettek <aljoscha@apache.org>
wrote:

> Hi Arvid,
> I have a fix that I hope fixes your problem:
> https://github.com/aljoscha/flink/tree/serializer-factories-fix
>
> Could you try building it and running your example?
>
> Cheers,
> Aljoscha
>
> On Fri, Jan 30, 2015 at 3:30 PM, Aljoscha Krettek <aljoscha@apache.org>
> wrote:
> > We have a bit of a divide in how we handle TypeSerializer and
> > TypeComparator: TypeSerializer does not handle duplication but relies
> > on outside code (RuntimeStatefulSerializerFactory) to perform the
> > duplication. TypeComparator does duplication itself. There is also the
> > RuntimeComparatorFactory. This, however, does not perform duplication
> > but simply hands out the same TypeComparator instance multiple times.
> > With comparators, the user of the comparator is responsible for
> > calling duplicate() on the comparator.
> >
> > I started work on a branch where I add a duplicate() method to
> > TypeSerializer that simply return itself for stateless serializers and
> > does a deep copy if it is stateful. The TypeSerializers no longer have
> > method isStateful() and I replaced the two serializer factories by one
> > factory that always returns a "duplicate" of the serializer it holds.
> >
> > I think we should consolidate the two approaches. So either let the
> > factories handle all the duplication or let the user of the
> > comparator/serializer always handle duplication. I think the former is
> > better, since the latter makes it very easy to forget to duplicate.
> >
> > On Fri, Jan 30, 2015 at 11:34 AM, Aljoscha Krettek <aljoscha@apache.org>
> wrote:
> >> Maybe we can get rid of the serializer factories altogether. We could
> >> enhance the Serializers with a method duplicate() that does nothing
> >> for stateless serializers and does a deep copy for stateful
> >> serializers. This would also consolidate all the knowledge about
> >> stateful/stateless in one place.
> >>
> >> On Fri, Jan 30, 2015 at 11:25 AM, Aljoscha Krettek <aljoscha@apache.org>
> wrote:
> >>> Of course it doesn't work. The ClassLoader is declared transient in
> >>> the serializer factory, so it is Null once the factory has been
> >>> serialized/deserialized once. I open a Jira issue:
> >>> https://issues.apache.org/jira/browse/FLINK-1463
> >>>
> >>> On Thu, Jan 29, 2015 at 9:43 PM, Stephan Ewen <sewen@apache.org>
> wrote:
> >>>> Thanks for reporting this, Arvid. I remember that we saw something
> similar
> >>>> a while back. I will take a look at this later today (I am in PST zone
> >>>> now), can hopefully fix this.
> >>>>
> >>>> Greetings,
> >>>> Stephan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message