flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <balassi.mar...@gmail.com>
Subject Re: Graph API Questions
Date Wed, 07 Jan 2015 10:54:54 GMT
+1

On Wed, Jan 7, 2015 at 10:12 AM, Stephan Ewen <sewen@apache.org> wrote:

> +1 as well
> Am 07.01.2015 10:04 schrieb "Vasiliki Kalavri" <vasilikikalavri@gmail.com
> >:
>
> > +1 makes sense :)
> >
> > On 7 January 2015 at 09:51, Kostas Tzoumas <ktzoumas@apache.org> wrote:
> >
> > > Perhaps we should have both code paths (Spargel and Spargel inside
> Graph
> > > API) in the code for a while to enable current Spargel users to migrate
> > > their applications before the breaking change.
> > >
> > > This means that you have Spargel inside Graph API as you plan to, but
> you
> > > do not remove the current Spargel classes in your PR.
> > >
> > > What do you think?
> > >
> > > Kostas
> > >
> > >
> > >
> > > On Tue, Jan 6, 2015 at 10:32 PM, Vasiliki Kalavri <
> > > vasilikikalavri@gmail.com
> > > > wrote:
> > >
> > > > Spargel will be included in the graph API.
> > > > We are already using spargel, but we will move its classes to the
> graph
> > > > API, change them to work with Vertex and Edge types and implement
> some
> > > > extensions. This is the last big change we need to make and then we
> can
> > > > include it in flink-addons :-)
> > > >
> > > > -V.
> > > >
> > > > On 6 January 2015 at 22:09, Henry Saputra <henry.saputra@gmail.com>
> > > wrote:
> > > >
> > > > > Just curious, is the new Graph APIs suppose to replace
> flink-spargel?
> > > > >
> > > > > - Henry
> > > > >
> > > > > On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
> > > > > <vasilikikalavri@gmail.com> wrote:
> > > > > > Hello Aljoscha and everyone,
> > > > > >
> > > > > > Andra and Daniel might also want to give input :)
> > > > > >
> > > > > > - As Kostas already said, we've had some issues with the
> > > TypeExtractor,
> > > > > but
> > > > > > these were mostly due to our lack of understanding of how it
> works.
> > > It
> > > > > > would be great if we could document the functionality and provide
> > > some
> > > > > tips
> > > > > > on usage of different methods. For example, when to use
> > > createTypeInfo
> > > > > and
> > > > > > when getReturnTypes? When is it necessary to implement
> > > > > ResultTypeQueryable?
> > > > > > etc.
> > > > > >
> > > > > > - An ugly/bothering issue is not being able to return a scalar
> > value
> > > > from
> > > > > > an operation and having to work with a single-value DataSet.
I
> > > > understand
> > > > > > this is not trivial to support and we are currently working
> around
> > > this
> > > > > > using broadcast variables.
> > > > > >
> > > > > > - The lack of a count operator (which is work in progress, no?)
> > > > > >
> > > > > > - Not being able to easily access the group key inside a
> > > reduce/coGroup
> > > > > > UDF. If I'm not mistaken, there was an issue/discussion on this
> as
> > > > well.
> > > > > >
> > > > > > If anything else comes to mind, I'll let you know!
> > > > > > Overall, I think we've had very few problems and nothing for
> which
> > we
> > > > > > (Stephan) couldn't find a workaround :-)
> > > > > >
> > > > > > Cheers,
> > > > > > -V.
> > > > > >
> > > > > > PS: I believe we'll be able to make the Graph API (code name
> Gelly)
> > > PR
> > > > > very
> > > > > > very soon ^^
> > > > > >
> > > > > > On 6 January 2015 at 11:24, Kostas Tzoumas <ktzoumas@apache.org>
> > > > wrote:
> > > > > >
> > > > > >> From what I have been "overhearing" from the Graph API
> > development,
> > > > some
> > > > > >> hurdles were type extractor errors that could not be deciphered
> > > > easily,
> > > > > and
> > > > > >> the inability to bring back data easily back to the client.
> > > > > >>
> > > > > >> But it would be great if Vasia could elaborate!
> > > > > >>
> > > > > >> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <
> > > > aljoscha@apache.org>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi,
> > > > > >> > I guess this is mostly directed at Vasia, since you
are the
> main
> > > > > >> > developer behind the graph API right now. (Correct
me if I'm
> > > wrong).
> > > > > >> >
> > > > > >> > Could you share some of the difficulties that you encountered
> > > while
> > > > > >> > developing the API and some of the stuff that bothers
you. I
> > want
> > > to
> > > > > >> > work on making our core API better for developing specialised
> > > APIs.
> > > > > >> >
> > > > > >> > Cheers,
> > > > > >> > Aljoscha
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message