flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: [Gelly] Re-organize / cleanup examples and library
Date Wed, 29 Jul 2015 12:22:11 GMT
Quick question: Should we actually create a single "flink-examples"
project, where we put all examples (batch, streaming, gelly) both Java and
Scala?

Would help us reduce the plethora of maven projects a bit ;-)

On Wed, Jul 29, 2015 at 2:20 PM, Andra Lungu <lungu.andra@gmail.com> wrote:

> Hi Vasia,
>
> We should document the examples in the Gelly guide, I totally agree,
> perhaps something similar to what Giraph offers.
> However, if you recall, the examples are also there for test purposes. The
> library methods contain just the run method; no main method. For the tests,
> the nicest way to do this is to call a main class... The way I see it,
> there is no reason to refactor something that works and that does not
> bother anyone. This is clearly not a bug!
>
> How about we focus on adding more interesting features like, for instance,
> graph partitioning and keep refactoring of the examples (changes of names,
> etc) for some other time?
>
> Cheers,
> Andra
>
> On Tue, Jul 28, 2015 at 7:15 PM, Stephan Ewen <sewen@apache.org> wrote:
>
> > Sounds very good, thanks Vasia!
> >
> > +1
> >
> > On Tue, Jul 28, 2015 at 5:48 PM, Till Rohrmann <trohrmann@apache.org>
> > wrote:
> >
> > > Sounds reasonable to me. +1
> > >
> > > On Tue, Jul 28, 2015 at 4:14 PM, Vasiliki Kalavri <
> > > vasilikikalavri@gmail.com
> > > > wrote:
> > >
> > > > Hello squirrels,
> > > >
> > > > adding Gelly examples has been a great way to let in new contributors
> > and
> > > > we have had quite a big number of contributions!
> > > > However, I think it's about time we re-organize and clean them up a
> > bit.
> > > >
> > > > We currently have 13 examples.
> > > >
> > > > - 5 examples simply use vertex-centric library methods, i.e. they are
> > > > almost identical and the only method they demonstrate is
> graph.run(new
> > > > LibraryAlgorithm()).
> > > >
> > > > I think it's enough if we keep 1 of those. Another problem is that
> the
> > > > implementations of these methods are "hidden" inside the library
> > > methods. I
> > > > propose we turn 1 of the 5 examples into showing how to use
> > > vertex-centric
> > > > iterations.
> > > >
> > > > - 3 examples show how to use the GSA iteration. We can also keep 1,
> > e.g.
> > > > SSSP.
> > > > The other 2 can be moved to the library.
> > > >
> > > > - I think the rest of the examples have a purpose and show how to use
> > > > different Gelly methods:
> > > >
> > > > 1. EuclideanGraphWeighing: use of triplets
> > > > 2. GraphMetrics: use of metrics and statistics methods
> > > > 3. IncrementalSSSP: use of message direction in iterations
> > > > 4. JaccardSimilarity: use of neighborhood methods and joins
> > > > 5. MusicProfiles: how to mix Gelly with the DataSet API
> > > >
> > > > To sum up, I propose we keep the above 5 examples, plus 1 to show how
> > to
> > > > use library methods, 1 for vertex-centric and 1 for GSA -> total 8
> and
> > > move
> > > > the redundant GSA implementations to the library.
> > > >
> > > > I believe we should also improve the examples documentation and refer
> > to
> > > > the functionality that each of them demonstrates.
> > > >
> > > > What do you think about this? If you agree, I will create a JIRA and
> do
> > > it!
> > > >
> > > > Cheers,
> > > > Vasia.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message