Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <CAJYNqUKsHGW11bXm-4_J78UfXJeNOWBZD74uue_7kjYXF31ToA@mail.gmail.com>
References: <CAJYNqUKsHGW11bXm-4_J78UfXJeNOWBZD74uue_7kjYXF31ToA@mail.gmail.com>
From: Matt <dromitlabs@gmail.com>
Date: Thu, 22 Dec 2016 18:17:08 -0300
Message-ID: <CAJYNqUJuPZ5gqTzAc-cKSpGJe0mdBvzG1PkOJ-tkTArSze=98Q@mail.gmail.com>
Subject: Re: Caching collected objects in .apply()
To: user@flink.apache.org
Content-Type: multipart/alternative; boundary=001a114e63c2c831ac054445cabe
archived-at: Thu, 22 Dec 2016 21:17:20 -0000

--001a114e63c2c831ac054445cabe
Content-Type: text/plain; charset=UTF-8

Just to be clear, the stream is of String elements. The first part of the
pipeline (up to the first .apply) receives those strings, and returns
objects of another class ("A" let's say).

On Thu, Dec 22, 2016 at 6:04 PM, Matt <dromitlabs@gmail.com> wrote:

> Hello,
>
> I have a window processing 10 objects at a time, and creating 1 as a
> result. The problem is in order to create that object I need the object
> from the previous window.
>
> I'm doing this:
>
> stream
>   .keyBy(...some key...)
>   .countWindow(10, 1)
>   .apply(...creates an element A...)
>   .keyBy(...same key as above...)
>   .countWindow(2, 1)
>   .apply(...updates A with the value of the previous element A...)
>   .addSink(...)
>
> Probably there is a way to retrieve the last collected object inside the
> first .apply(), or to cache it somehow.
>
> Is there a better way to achieve the same? How inefficient is this?
>
> Regards,
> Matt
>

--001a114e63c2c831ac054445cabe
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Just to be clear, the stream is of String elements. The fi=
rst part of the pipeline (up to the first .apply) receives those strings, a=
nd returns objects of another class (&quot;A&quot; let&#39;s say).</div><di=
v class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Thu, Dec 22, 2016=
 at 6:04 PM, Matt <span dir=3D"ltr">&lt;<a href=3D"mailto:dromitlabs@gmail.=
com" target=3D"_blank">dromitlabs@gmail.com</a>&gt;</span> wrote:<br><block=
quote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc=
 solid;padding-left:1ex"><div dir=3D"ltr">Hello,<div><br></div><div>I have =
a window processing 10 objects at a time, and creating 1 as a result. The p=
roblem is in order to create that object I need the object from the previou=
s window.</div><div><br></div><div>I&#39;m doing this:</div><div><br></div>=
<div>stream<br></div><div>=C2=A0 .keyBy(...some key...)</div><div>=C2=A0 .c=
ountWindow(10, 1)</div><div>=C2=A0=C2=A0.apply(...creates an element A...)<=
/div><div>=C2=A0=C2=A0.keyBy(...same key as above...)</div><div>=C2=A0=C2=
=A0.countWindow(2, 1)</div><div>=C2=A0=C2=A0.apply(...updates A with the va=
lue of the previous element A...)</div><div>=C2=A0 .addSink(...)</div><div>=
<br></div><div>Probably there is a way to retrieve the last collected objec=
t inside the first .apply(), or to cache it somehow.<br></div><div><br></di=
v><div>Is there a better way to achieve the same? How inefficient is this?<=
/div><div><br></div><div>Regards,</div><div>Matt</div></div>
</blockquote></div><br></div>

--001a114e63c2c831ac054445cabe--