flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Graph API Questions
Date Wed, 07 Jan 2015 09:12:11 GMT
+1 as well
Am 07.01.2015 10:04 schrieb "Vasiliki Kalavri" <vasilikikalavri@gmail.com>:

> +1 makes sense :)
>
> On 7 January 2015 at 09:51, Kostas Tzoumas <ktzoumas@apache.org> wrote:
>
> > Perhaps we should have both code paths (Spargel and Spargel inside Graph
> > API) in the code for a while to enable current Spargel users to migrate
> > their applications before the breaking change.
> >
> > This means that you have Spargel inside Graph API as you plan to, but you
> > do not remove the current Spargel classes in your PR.
> >
> > What do you think?
> >
> > Kostas
> >
> >
> >
> > On Tue, Jan 6, 2015 at 10:32 PM, Vasiliki Kalavri <
> > vasilikikalavri@gmail.com
> > > wrote:
> >
> > > Spargel will be included in the graph API.
> > > We are already using spargel, but we will move its classes to the graph
> > > API, change them to work with Vertex and Edge types and implement some
> > > extensions. This is the last big change we need to make and then we can
> > > include it in flink-addons :-)
> > >
> > > -V.
> > >
> > > On 6 January 2015 at 22:09, Henry Saputra <henry.saputra@gmail.com>
> > wrote:
> > >
> > > > Just curious, is the new Graph APIs suppose to replace flink-spargel?
> > > >
> > > > - Henry
> > > >
> > > > On Tue, Jan 6, 2015 at 3:51 AM, Vasiliki Kalavri
> > > > <vasilikikalavri@gmail.com> wrote:
> > > > > Hello Aljoscha and everyone,
> > > > >
> > > > > Andra and Daniel might also want to give input :)
> > > > >
> > > > > - As Kostas already said, we've had some issues with the
> > TypeExtractor,
> > > > but
> > > > > these were mostly due to our lack of understanding of how it works.
> > It
> > > > > would be great if we could document the functionality and provide
> > some
> > > > tips
> > > > > on usage of different methods. For example, when to use
> > createTypeInfo
> > > > and
> > > > > when getReturnTypes? When is it necessary to implement
> > > > ResultTypeQueryable?
> > > > > etc.
> > > > >
> > > > > - An ugly/bothering issue is not being able to return a scalar
> value
> > > from
> > > > > an operation and having to work with a single-value DataSet. I
> > > understand
> > > > > this is not trivial to support and we are currently working around
> > this
> > > > > using broadcast variables.
> > > > >
> > > > > - The lack of a count operator (which is work in progress, no?)
> > > > >
> > > > > - Not being able to easily access the group key inside a
> > reduce/coGroup
> > > > > UDF. If I'm not mistaken, there was an issue/discussion on this as
> > > well.
> > > > >
> > > > > If anything else comes to mind, I'll let you know!
> > > > > Overall, I think we've had very few problems and nothing for which
> we
> > > > > (Stephan) couldn't find a workaround :-)
> > > > >
> > > > > Cheers,
> > > > > -V.
> > > > >
> > > > > PS: I believe we'll be able to make the Graph API (code name Gelly)
> > PR
> > > > very
> > > > > very soon ^^
> > > > >
> > > > > On 6 January 2015 at 11:24, Kostas Tzoumas <ktzoumas@apache.org>
> > > wrote:
> > > > >
> > > > >> From what I have been "overhearing" from the Graph API
> development,
> > > some
> > > > >> hurdles were type extractor errors that could not be deciphered
> > > easily,
> > > > and
> > > > >> the inability to bring back data easily back to the client.
> > > > >>
> > > > >> But it would be great if Vasia could elaborate!
> > > > >>
> > > > >> On Tue, Jan 6, 2015 at 10:39 AM, Aljoscha Krettek <
> > > aljoscha@apache.org>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi,
> > > > >> > I guess this is mostly directed at Vasia, since you are
the main
> > > > >> > developer behind the graph API right now. (Correct me if
I'm
> > wrong).
> > > > >> >
> > > > >> > Could you share some of the difficulties that you encountered
> > while
> > > > >> > developing the API and some of the stuff that bothers you.
I
> want
> > to
> > > > >> > work on making our core API better for developing specialised
> > APIs.
> > > > >> >
> > > > >> > Cheers,
> > > > >> > Aljoscha
> > > > >> >
> > > > >>
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message