Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Received-SPF: pass (athena.apache.org: domain of ted.dunning@gmail.com
 designates 209.85.223.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CANC1h_t6_eMiQRNcHQoHr-xEng-oaWNdUqLn+qTDgLcvbqYM0w@mail.gmail.com>
References: 
 <CANMXwW0PwkzyKzOrVPHODwfuE3ekWWV9ZWkw6k1-mnxTyYJZLw@mail.gmail.com>
 <CANC1h_t6_eMiQRNcHQoHr-xEng-oaWNdUqLn+qTDgLcvbqYM0w@mail.gmail.com>
From: Ted Dunning <ted.dunning@gmail.com>
Date: Sat, 28 Feb 2015 07:57:13 +0100
Message-ID: 
 <CAJwFCa3+ModYkeBL3nXukFsUyCtCf4CULyXjwYOVEj-oxu62aA@mail.gmail.com>
Subject: Re: Thoughts About Object Reuse and Collection Execution
To: "dev@flink.apache.org" <dev@flink.apache.org>
Content-Type: multipart/alternative; boundary=20cf302920ea6dd274051020803c

--20cf302920ea6dd274051020803c
Content-Type: text/plain; charset=UTF-8

This is going to have profound performance implications if this is the only
path for iteration.


On Fri, Feb 27, 2015 at 10:58 PM, Stephan Ewen <sewen@apache.org> wrote:

> I vote to have the key extractor return a new value each time. That means
> that objects are not reused everywhere where it is possible, but still in
> most places, which still helps.
>
> What still puzzles me: I thought that the collection execution stores
> copies of the returned records by default (reuse safe mode).
> Am 27.02.2015 15:36 schrieb "Aljoscha Krettek" <aljoscha@apache.org>:
>
> > Hello Nation of Flink,
> > while figuring out this bug:
> > https://issues.apache.org/jira/browse/FLINK-1569
> > I came upon some difficulties. The problem is that the
> > KeyExtractorMappers always
> > return the same tuple. This is problematic, since Collection Execution
> > does simply store the returned values in a list. These elements are
> > not copied before they are stored when object reuse is enabled.
> > Therefore, the whole list will contain only that one reused element.
> >
> > I see two options for solving this:
> > 1. Change KeyExtractorMappers to always return a new tuple, thereby
> > making object-reuse mode in cluster execution useless for key
> > extractors.
> >
> > 2. Change collection execution mapper to always make copies of the
> > returned elements. This would make object-reuse in collection
> > execution pretty much obsolete, IMHO.
> >
> > How should we proceed with this?
> >
> > Cheers,
> > Aljoscha
> >
>

--20cf302920ea6dd274051020803c--