flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: [Gelly] Re-organize / cleanup examples and library
Date Wed, 29 Jul 2015 12:33:12 GMT
+1 makes also sense to me, Vasia.

+1 for one example project but let's also create a staging examples
project. Otherwise things might get mixed up.

On Wed, Jul 29, 2015 at 2:28 PM, Andra Lungu <lungu.andra@gmail.com> wrote:

> Makes perfect sense, Stephan, as long as there is a separate folder for
> each (stating the obvious :) ).
> +1
>
> On Wed, Jul 29, 2015 at 2:22 PM, Stephan Ewen <sewen@apache.org> wrote:
>
> > Quick question: Should we actually create a single "flink-examples"
> > project, where we put all examples (batch, streaming, gelly) both Java
> and
> > Scala?
> >
> > Would help us reduce the plethora of maven projects a bit ;-)
> >
> > On Wed, Jul 29, 2015 at 2:20 PM, Andra Lungu <lungu.andra@gmail.com>
> > wrote:
> >
> > > Hi Vasia,
> > >
> > > We should document the examples in the Gelly guide, I totally agree,
> > > perhaps something similar to what Giraph offers.
> > > However, if you recall, the examples are also there for test purposes.
> > The
> > > library methods contain just the run method; no main method. For the
> > tests,
> > > the nicest way to do this is to call a main class... The way I see it,
> > > there is no reason to refactor something that works and that does not
> > > bother anyone. This is clearly not a bug!
> > >
> > > How about we focus on adding more interesting features like, for
> > instance,
> > > graph partitioning and keep refactoring of the examples (changes of
> > names,
> > > etc) for some other time?
> > >
> > > Cheers,
> > > Andra
> > >
> > > On Tue, Jul 28, 2015 at 7:15 PM, Stephan Ewen <sewen@apache.org>
> wrote:
> > >
> > > > Sounds very good, thanks Vasia!
> > > >
> > > > +1
> > > >
> > > > On Tue, Jul 28, 2015 at 5:48 PM, Till Rohrmann <trohrmann@apache.org
> >
> > > > wrote:
> > > >
> > > > > Sounds reasonable to me. +1
> > > > >
> > > > > On Tue, Jul 28, 2015 at 4:14 PM, Vasiliki Kalavri <
> > > > > vasilikikalavri@gmail.com
> > > > > > wrote:
> > > > >
> > > > > > Hello squirrels,
> > > > > >
> > > > > > adding Gelly examples has been a great way to let in new
> > contributors
> > > > and
> > > > > > we have had quite a big number of contributions!
> > > > > > However, I think it's about time we re-organize and clean them
> up a
> > > > bit.
> > > > > >
> > > > > > We currently have 13 examples.
> > > > > >
> > > > > > - 5 examples simply use vertex-centric library methods, i.e.
they
> > are
> > > > > > almost identical and the only method they demonstrate is
> > > graph.run(new
> > > > > > LibraryAlgorithm()).
> > > > > >
> > > > > > I think it's enough if we keep 1 of those. Another problem is
> that
> > > the
> > > > > > implementations of these methods are "hidden" inside the library
> > > > > methods. I
> > > > > > propose we turn 1 of the 5 examples into showing how to use
> > > > > vertex-centric
> > > > > > iterations.
> > > > > >
> > > > > > - 3 examples show how to use the GSA iteration. We can also
keep
> 1,
> > > > e.g.
> > > > > > SSSP.
> > > > > > The other 2 can be moved to the library.
> > > > > >
> > > > > > - I think the rest of the examples have a purpose and show how
to
> > use
> > > > > > different Gelly methods:
> > > > > >
> > > > > > 1. EuclideanGraphWeighing: use of triplets
> > > > > > 2. GraphMetrics: use of metrics and statistics methods
> > > > > > 3. IncrementalSSSP: use of message direction in iterations
> > > > > > 4. JaccardSimilarity: use of neighborhood methods and joins
> > > > > > 5. MusicProfiles: how to mix Gelly with the DataSet API
> > > > > >
> > > > > > To sum up, I propose we keep the above 5 examples, plus 1 to
show
> > how
> > > > to
> > > > > > use library methods, 1 for vertex-centric and 1 for GSA ->
total
> 8
> > > and
> > > > > move
> > > > > > the redundant GSA implementations to the library.
> > > > > >
> > > > > > I believe we should also improve the examples documentation
and
> > refer
> > > > to
> > > > > > the functionality that each of them demonstrates.
> > > > > >
> > > > > > What do you think about this? If you agree, I will create a
JIRA
> > and
> > > do
> > > > > it!
> > > > > >
> > > > > > Cheers,
> > > > > > Vasia.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message