Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
In-Reply-To: <8B754047F81D6B4290B9F4CE928333A517C6D12A@lhreml503-mbx>
References: <8B754047F81D6B4290B9F4CE928333A517C6AFE6@lhreml503-mbx>
 <FBDE95D1-3C78-4F64-8A21-6715954C17FF@apache.org> <8B754047F81D6B4290B9F4CE928333A517C6CAC8@lhreml503-mbx>
 <6F8FD537-351D-47C6-9CCA-BE4B04D06AF5@data-artisans.com> <8B754047F81D6B4290B9F4CE928333A517C6D12A@lhreml503-mbx>
From: Fabian Hueske <fhueske@gmail.com>
Date: Thu, 18 May 2017 15:44:28 +0200
Message-ID: <CAAdrtT2kBvJ0ERWL5d++NnR5dFAnUmAoTC8zcXBLJVR2Ayp3_g@mail.gmail.com>
Subject: Re: ListState to List
To: "dev@flink.apache.org" <dev@flink.apache.org>
Content-Type: multipart/alternative; boundary="001a113dddd2584247054fcc9cc0"
archived-at: Thu, 18 May 2017 13:45:09 -0000

--001a113dddd2584247054fcc9cc0
Content-Type: text/plain; charset="UTF-8"

I think the ListState interface is pretty well suited for this job.
It allows to add elements with low effort and can serve all elements of a
list through an iterator. Depending on the implementation the elements
could be deserialized as needed.
If the user code needs a List with all elements, it won't make a big
performance difference whether the state backend or the user code copies
the elements into a List object.

2017-05-18 15:24 GMT+02:00 Radu Tudoran <radu.tudoran@huawei.com>:

> Actually that was one option that I was considering. I am still a bit
> fuzzy about the advantages and disadvantages of using one type of state
> over another. I know that using ValueState would mean that when getting the
> object value (i.e. a List in this case) the whole would be deserialized at
> once. This is ok if I need to go anyway through all elements. However, I
> understand that when I need to update the state the same will hold - the
> whole list would be serialized and re-write instead of a single element.
> Therefore if we want to get always the best performances it seemed to me
> that it would be worth considering have many specialized types of states -
> hence my proposal.
>
>
>
> -----Original Message-----
> From: Kostas Kloudas [mailto:k.kloudas@data-artisans.com]
> Sent: Thursday, May 18, 2017 11:49 AM
> To: dev@flink.apache.org
> Subject: Re: ListState to List
>
> Hi Radu,
>
> Why not using a ValueState that inside stored the whole list.
> Whenever you state#get() you get the whole list and you can sort it.
>
> Kostas
>
> > On May 18, 2017, at 3:31 AM, Radu Tudoran <radu.tudoran@huawei.com>
> wrote:
> >
> > Hi Aljoscha,
> >
> > Thanks for the clarification. I understand that there might be
> advantages in some cases not to have the List-like interface, while in
> other scenarios (like the one I described there aren't). Considering this,
> why not having 2 type of states: ListState and StreamInListState - users
> would use the one it is more appropriate. What do you think?
> >
> > -----Original Message-----
> > From: Aljoscha Krettek [mailto:aljoscha@apache.org]
> > Sent: Thursday, May 18, 2017 12:15 AM
> > To: dev@flink.apache.org
> > Subject: Re: ListState to List
> >
> > Hi,
> > The interface is restrictive on purpose because depending on the state
> backend it might not be possible to provide a List-like interface. There
> might be state backends that stream in the list from somewhere else or
> other restrictions. If we now allowed a more general interface here we
> would possibly prevent optimisations in the future or make certain
> implementations very hard to to efficiently.
> >
> > Best,
> > Aljoscha
> >
> >> On 16. May 2017, at 21:56, Radu Tudoran <radu.tudoran@huawei.com>
> wrote:
> >>
> >> Hi,
> >>
> >> I would like to work with ListState, more specifically I would need to
> access the contents and sort them. For this I would need a collection type
> (e.g., the List, Array...).
> >> However, I see that if I have a variable of type <<ListState state=..>>
> the only interfaces I have are:
> >> state.get -> which returns an Iterable Or state.get.getIterator which
> >> returns an Iterator
> >>
> >> Basically if I use any of these I need now to copy the contents in an
> actual List of Array.  Is there any way to avoid this? ..perhaps there is
> an implicit type that I can convert to...
> >>
> >
>
>

--001a113dddd2584247054fcc9cc0--