flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: [stratosphere-dev] Spark comparison
Date Sat, 30 Aug 2014 16:25:45 GMT
Hi!

I agree with Kostas, the code base of Stratosphere that was used was quite
old.

The current Flnk version is different already, with the new APIs and
different type handling.

Flink is taking a route that makes sure that the runtime is very robust,
memory wise. We pay currently a few CPU cycles overhead for that, but we
have an effort gong to bring that down.

It would be interesting to rerun the experiments then...

Greetings,
Stephan



On Sat, Aug 30, 2014 at 9:16 AM, Kostas Tzoumas <ktzoumas@apache.org> wrote:

> Hi Anirvan,
>
> Yes, I am familiar with this thesis. I think that this comparison is by now
> quite old (>1 year if I am not mistaken), and both systems have evolved
> substantially since then.
>
> Kostas
>
>
> On Fri, Aug 29, 2014 at 7:01 PM, Robert Metzger <rmetzger@apache.org>
> wrote:
>
> > Forwarding the message to the new mailing list ...
> >
> > ---------- Forwarded message ----------
> > From: Nirvanesque <nirvanesque.paris@gmail.com>
> > Date: Fri, Aug 29, 2014 at 1:57 PM
> > Subject: Re: [stratosphere-dev] Spark comparison
> > To: stratosphere-dev@googlegroups.com
> >
> >
> > Ufuk and the Flink team,
> >
> > You and your team are familiar by now with this comparison (Master thesis
> > of Ze Ni in the KTH Institute)
> > http://www.diva-portal.org/smash/get/diva2:605106/FULLTEXT01.pdf
> >
> > I would like to know your viewpoints in this direction?
> >
> > Thanks in advance,
> > Anirvan
> >
> >
> >
> > On Tuesday, December 3, 2013 6:19:57 PM UTC+1, Ufuk Celebi wrote:
> >
> > > Hey Ankur,
> > >
> > > I like the idea of a comparison matrix. We tried to do something
> similar
> > > with Hadoop already (parts of it are on the front page of our website),
> > > which we used for a local summit here. Comparing Stratosphere to Spark
> in
> > > this way would be a natural extension to this. ;-)
> > >
> > > Internally, we ran some benchmarks against 0.7.3 (unfortunately right
> > > before the 0.8 release). We didn't publish the results as there are
> > certain
> > > aspects that make the comparison unfair (for example we have no fault
> > > tolerance right now whereas Spark does). As soon as we (re-)introduce
> > fault
> > > tolerance mechanisms, we will re-run the benchmarks.
> > >
> > > I can publish the code for the Stratosphere and Spark programs we
> looked
> > > at on GitHub. If I add Scala versions of the Stratosphere programs,
> this
> > > will also go to your proposed direction of having a direct comparison.
> > >
> > > Is there any specific use case where you want to see numbers? Or is it
> > > more like a general thing where you want to see how both systems
> perform?
> > >
> > > Best,
> > >
> > > Ufuk
> > >
> > > On 03 Dec 2013, at 18:03, Ankur Chauhan <an...@malloc64.com> wrote:
> > >
> > > Hi all,
> > >
> > >
> > > Sitting at spark-summit 2013, I was interested in figuring out if
> anyone
> > > has done a feature comparison and or benchmarks against
> spark/storm/etc.
> > > This may also serve as a "compatibility matrix" and would help a lot
> when
> > > people want to compare the two projects and help us understand what are
> > the
> > > strengths and weakness of each project.
> > >
> > > -- Ankur
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "stratosphere-dev" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> an
> > > email to stratosphere-d...@googlegroups.com.
> > >
> > > Visit this group at http://groups.google.com/group/stratosphere-dev.
> > > For more options, visit https://groups.google.com/groups/opt_out.
> > >
> > >
> > >  --
> > You received this message because you are subscribed to the Google Groups
> > "stratosphere-dev" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to stratosphere-dev+unsubscribe@googlegroups.com.
> > Visit this group at http://groups.google.com/group/stratosphere-dev.
> > For more options, visit https://groups.google.com/d/optout.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message