Return-Path: X-Original-To: apmail-flink-dev-archive@www.apache.org Delivered-To: apmail-flink-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B885511C42 for ; Sun, 14 Sep 2014 09:15:27 +0000 (UTC) Received: (qmail 4447 invoked by uid 500); 14 Sep 2014 09:15:27 -0000 Delivered-To: apmail-flink-dev-archive@flink.apache.org Received: (qmail 4378 invoked by uid 500); 14 Sep 2014 09:15:27 -0000 Mailing-List: contact dev-help@flink.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.incubator.apache.org Delivered-To: mailing list dev@flink.incubator.apache.org Received: (qmail 4367 invoked by uid 99); 14 Sep 2014 09:15:27 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 14 Sep 2014 09:15:27 +0000 X-ASF-Spam-Status: No, hits=-1999.5 required=5.0 tests=ALL_TRUSTED,HTML_MESSAGE,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Sun, 14 Sep 2014 09:14:53 +0000 Received: (qmail 4285 invoked by uid 99); 14 Sep 2014 09:14:48 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 14 Sep 2014 09:14:48 +0000 Received: from localhost (HELO mail-qg0-f42.google.com) (127.0.0.1) (smtp-auth username ktzoumas, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Sun, 14 Sep 2014 09:14:47 +0000 Received: by mail-qg0-f42.google.com with SMTP id q107so2698508qgd.15 for ; Sun, 14 Sep 2014 02:14:46 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.224.29.195 with SMTP id r3mr25553936qac.47.1410686086760; Sun, 14 Sep 2014 02:14:46 -0700 (PDT) Received: by 10.96.97.132 with HTTP; Sun, 14 Sep 2014 02:14:46 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Sep 2014 11:14:46 +0200 Message-ID: Subject: Re: Scala API rewrite almost complete From: Kostas Tzoumas To: "dev@flink.incubator.apache.org" Content-Type: multipart/alternative; boundary=047d7bf0c37e176d47050302f38f X-Virus-Checked: Checked by ClamAV on apache.org --047d7bf0c37e176d47050302f38f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Good catch, I suggest to use examples On Sat, Sep 13, 2014 at 3:27 PM, M=C3=A1rton Balassi wrote: > Pull request issued. One minor naming concern: > > As of today the scala examples are located at > the org.apache.flink.examples.scala package, while the java ones in > the org.apache.flink.example.java. I suggest using only one convention fo= r > this either example or examples. > > Cheers, > > Marton > > On Fri, Sep 12, 2014 at 9:17 PM, M=C3=A1rton Balassi > wrote: > > > Sorry for being a bit silent after already bidding on LR. The pull > request > > is coming soon. > > > > On Fri, Sep 12, 2014 at 6:25 PM, Stephan Ewen wrote: > > > >> I suppose that having the option between simple return type, and a > >> collector is the easiest to understand. > >> Am 12.09.2014 16:50 schrieb "Aljoscha Krettek" : > >> > >> > So, should I change join and coGroup to have a simple return value, = no > >> > Option or Collection? Also what's happening with the relational > >> > examples and the LinearRegression examples? I'd like to make a pull > >> > request before this weekend. > >> > > >> > I also added a test that checks whether the Scala API has the same > >> > methods as the Java API (ScalaAPICompletenessTest). > >> > > >> > On Fri, Sep 12, 2014 at 4:00 PM, Aljoscha Krettek < > aljoscha@apache.org> > >> > wrote: > >> > > Yes, there is already a Collector version, you can do: > >> > > > >> > > left.join(right).where("foo").equalTo("bar") { > >> > > (left, right, out: Collector[Page]) =3D> > >> > > if (...) out.collect(...) > >> > > } > >> > > > >> > > I wasn't sure on what our Function2 variant should be. That's why = I > >> > > asked. There are some cases where you want to have the option (pun > >> > > intended) of either returning something from a join or not. But yo= u > >> > > could also use a filter after the join. It's a preference thing, I > >> > > suppose. > >> > > > >> > > On Fri, Sep 12, 2014 at 2:43 PM, Stephan Ewen > >> wrote: > >> > >> I think it seems weird that normal joins need to go through optio= n. > >> > >> > >> > >> The option variant is to allow filters in the join function. > >> Wouldn't a > >> > >> collector variant allow you to do the same, and would be function= 3 > ? > >> I > >> > know > >> > >> that option reads more functionally... > >> > >> > >> > >> > >> > >> > >> > >> Am 12.09.2014 14:24 schrieb "Aljoscha Krettek" < > aljoscha@apache.org > >> >: > >> > >>> > >> > >>> As already mentioned this is not possible because of type erasur= e. > >> We > >> > >>> can only have one join variant that takes a Function2. > >> > >>> > >> > >>> On Fri, Sep 12, 2014 at 12:34 PM, Stephan Ewen > >> > wrote: > >> > >>> > It would be nice to have a join variant that directly returns > the > >> > value > >> > >>> > rathern than an option. Why not have both (they are wrapped as > >> > flatJoins > >> > >>> > anyway below, right?) > >> > >>> > > >> > >>> > On Fri, Sep 12, 2014 at 11:50 AM, Fabian Hueske < > >> fhueske@apache.org> > >> > >> wrote: > >> > >>> > > >> > >>> >> Sweet! I'm lovin' this :-) > >> > >>> >> > >> > >>> >> 2014-09-12 11:46 GMT+02:00 Aljoscha Krettek < > aljoscha@apache.org > >> >: > >> > >>> >> > >> > >>> >> > Also, you can use CaseClasses directly as the type for CSV > >> input. > >> > So > >> > >>> >> > instead of reading it as tuples and then having a mapper th= at > >> > maps to > >> > >>> >> > your case classes you can use: > >> > >>> >> > > >> > >>> >> > env.readCsv[Edge](...) > >> > >>> >> > > >> > >>> >> > On Fri, Sep 12, 2014 at 11:43 AM, Aljoscha Krettek < > >> > >> aljoscha@apache.org> > >> > >>> >> > wrote: > >> > >>> >> > > I added support for specifying keys by name for > CaseClasses. > >> > Check > >> > >> out > >> > >>> >> > > the PageRank and TriangleEnumeration examples to see it i= n > >> > action. > >> > >>> >> > > > >> > >>> >> > > @Kostas: I think you could use them for the TPC-H example= s. > >> > >>> >> > > > >> > >>> >> > > On Fri, Sep 12, 2014 at 7:23 AM, Aljoscha Krettek < > >> > >> aljoscha@apache.org > >> > >>> >> > > >> > >>> >> > wrote: > >> > >>> >> > >> Yes, that would allow list comprehensions. It would be > >> > possible to > >> > >>> >> > >> have the Collection signature for join (and coGroup), > i.e.: > >> > >>> >> > >> > >> > >>> >> > >> apply[R]((T, O) =3D> TraversableOnce[O]): DataSet[O] > >> > >>> >> > >> > >> > >>> >> > >> (T and O are the left and right input type, R is result > >> type) > >> > >>> >> > >> > >> > >>> >> > >> Then you can return collections and still return an > option, > >> as > >> > in: > >> > >>> >> > >> > >> > >>> >> > >> a.join(b).where(0).equalTo(0) { (l, r) =3D> if (r > ...) > >> Some(l) > >> > >> else > >> > >>> >> > None } > >> > >>> >> > >> > >> > >>> >> > >> Because there is an implicit conversion from Options to = a > >> > >> Collection. > >> > >>> >> > >> This will always wrap the return value in a List with on= ly > >> one > >> > >> value. > >> > >>> >> > >> I'm not sure we want the overhead here. I'm also not sur= e > >> > whether > >> > >> we > >> > >>> >> > >> want the overhead of always having to use an Option even > >> though > >> > >> the > >> > >>> >> > >> join always returns a value. > >> > >>> >> > >> > >> > >>> >> > >> What do you think? > >> > >>> >> > >> > >> > >>> >> > >> On Thu, Sep 11, 2014 at 11:22 PM, Fabian Hueske < > >> > >> fhueske@apache.org> > >> > >>> >> > wrote: > >> > >>> >> > >>> Hmmm, tricky question... > >> > >>> >> > >>> How about the Option for Join as this is a tuple-wise > >> > operation > >> > >> and > >> > >>> >> the > >> > >>> >> > >>> Collection for Cogroup which is group-wise? > >> > >>> >> > >>> Could we in that case use list comprehensions in Cogrou= p > >> > >> functions? > >> > >>> >> > >>> > >> > >>> >> > >>> Or is that too much mixing? > >> > >>> >> > >>> > >> > >>> >> > >>> 2014-09-11 23:00 GMT+02:00 Aljoscha Krettek < > >> > aljoscha@apache.org > >> > >>>: > >> > >>> >> > >>> > >> > >>> >> > >>>> I didn't look at the example either. > >> > >>> >> > >>>> > >> > >>> >> > >>>> Addings collections is easy, it's just that we can > either > >> > have > >> > >>> >> > >>>> Collections or the Option, not both. > >> > >>> >> > >>>> > >> > >>> >> > >>>> For the coding style I followed this: > >> > >>> >> > >>>> > >> > >>> >> > > >> > >> > >> > > >> > https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide > >> > >>> >> , > >> > >>> >> > >>>> which itself is based on this: > >> > http://docs.scala-lang.org/style/ > >> > >> . > >> > >>> >> It > >> > >>> >> > >>>> is different from the Java Code Guidelines we have in > >> place, > >> > >> yes. > >> > >>> >> > >>>> > >> > >>> >> > >>>> On Thu, Sep 11, 2014 at 10:10 PM, Fabian Hueske < > >> > >> fhueske@apache.org > >> > >>> >> > > >> > >>> >> > >>>> wrote: > >> > >>> >> > >>>> > I haven't looked at the LineRank example in detail, > but > >> if > >> > you > >> > >>> >> > think that > >> > >>> >> > >>>> > it adds something new to the examples collection, we > can > >> > >> certainly > >> > >>> >> > port > >> > >>> >> > >>>> it > >> > >>> >> > >>>> > also to Java. > >> > >>> >> > >>>> > I think the Option and Collector return types are > >> > sufficient > >> > >> right > >> > >>> >> > now > >> > >>> >> > >>>> but > >> > >>> >> > >>>> > if Collections are easy to add, go for it. ;-) > >> > >>> >> > >>>> > > >> > >>> >> > >>>> > Great that the Scala primitives are working! Also > thanks > >> > for > >> > >>> >> adding > >> > >>> >> > >>>> > genSequence and adapting my examples. > >> > >>> >> > >>>> > Btw. does the codestyle not apply for Scala files or > do > >> we > >> > >> have a > >> > >>> >> > >>>> different > >> > >>> >> > >>>> > there? > >> > >>> >> > >>>> > > >> > >>> >> > >>>> > 2014-09-11 17:55 GMT+02:00 Aljoscha Krettek < > >> > >> aljoscha@apache.org > >> > >>> >> >: > >> > >>> >> > >>>> > > >> > >>> >> > >>>> >> What about the LineRank example? We had that in Sca= la > >> but > >> > >> never > >> > >>> >> > had a > >> > >>> >> > >>>> >> Java Example. > >> > >>> >> > >>>> >> > >> > >>> >> > >>>> >> On Thu, Sep 11, 2014 at 5:51 PM, Aljoscha Krettek < > >> > >>> >> > aljoscha@apache.org> > >> > >>> >> > >>>> >> wrote: > >> > >>> >> > >>>> >> > Yes, I like that. For the ITCases I always just > >> copied > >> > the > >> > >> Java > >> > >>> >> > >>>> ITCase. > >> > >>> >> > >>>> >> > > >> > >>> >> > >>>> >> > The only examples that are missing now are > >> > >> LinearRegression and > >> > >>> >> > the > >> > >>> >> > >>>> >> > relational stuff. > >> > >>> >> > >>>> >> > > >> > >>> >> > >>>> >> > On Thu, Sep 11, 2014 at 5:48 PM, Fabian Hueske < > >> > >>> >> > fhueske@apache.org> > >> > >>> >> > >>>> >> wrote: > >> > >>> >> > >>>> >> >> I just removed the old CountEdgeDegrees example. > >> > >>> >> > >>>> >> >> That was a preprocessing step for the > >> > >> TriangleEnumeration, and > >> > >>> >> > is now > >> > >>> >> > >>>> >> part > >> > >>> >> > >>>> >> >> of the new TriangleEnumerationOpt example. > >> > >>> >> > >>>> >> >> So I guess, we don't need to port that one. As I > >> said > >> > >> before, > >> > >>> >> > I'd > >> > >>> >> > >>>> >> prefer to > >> > >>> >> > >>>> >> >> keep Java and Scala examples in sync. > >> > >>> >> > >>>> >> >> > >> > >>> >> > >>>> >> >> Cheers, Fabian > >> > >>> >> > >>>> >> >> > >> > >>> >> > >>>> >> >> 2014-09-11 17:40 GMT+02:00 Aljoscha Krettek < > >> > >>> >> > aljoscha@apache.org>: > >> > >>> >> > >>>> >> >> > >> > >>> >> > >>>> >> >>> I added the PageRank example, thanks again > fabian. > >> :D > >> > >>> >> > >>>> >> >>> > >> > >>> >> > >>>> >> >>> Regarding the other stuff: > >> > >>> >> > >>>> >> >>> - There is a comment in DataSet.scala about > >> including > >> > >>> >> > >>>> >> >>> org.apache.flink.api.scala._ because of the > >> > >> TypeInformation. > >> > >>> >> > >>>> >> >>> - I added generateSequence to > >> ExecutionEnvironment. > >> > >>> >> > >>>> >> >>> - It is possible to use Scala Primitives in > >> Array, I > >> > >> noticed > >> > >>> >> > it > >> > >>> >> > >>>> while > >> > >>> >> > >>>> >> >>> writing the tests, you probably had an older > >> version > >> > of > >> > >> the > >> > >>> >> > code. > >> > >>> >> > >>>> >> >>> - Yes, using List and other Interfaces is not > >> > possible, > >> > >> this > >> > >>> >> > is > >> > >>> >> > >>>> also > >> > >>> >> > >>>> >> >>> a restriction in the Java API. > >> > >>> >> > >>>> >> >>> > >> > >>> >> > >>>> >> >>> What do you think about the interface of join a= nd > >> > >> coGroup? > >> > >>> >> > Right > >> > >>> >> > >>>> now, > >> > >>> >> > >>>> >> >>> you can either use a lambda that returns an > Option > >> or > >> > the > >> > >>> >> > lambda > >> > >>> >> > >>>> with > >> > >>> >> > >>>> >> >>> the Collector. Originally I wanted to have also > >> have a > >> > >> lambda > >> > >>> >> > that > >> > >>> >> > >>>> >> >>> returns a Collection, but due to type erasure > this > >> has > >> > >> the > >> > >>> >> > same type > >> > >>> >> > >>>> >> >>> as the lambda with the Option so I couldn't use > it. > >> > >> There is > >> > >>> >> an > >> > >>> >> > >>>> >> >>> implicit conversion from Option to a Collection= , > >> so I > >> > >> could > >> > >>> >> > change > >> > >>> >> > >>>> it > >> > >>> >> > >>>> >> >>> without breaking the examples we have now. What > do > >> you > >> > >> think? > >> > >>> >> > >>>> >> >>> > >> > >>> >> > >>>> >> >>> So far we have ported: WordCount, KMeans, > >> > >>> >> ConnectedComponents, > >> > >>> >> > >>>> >> >>> WebLogAnalysis, TransitiveClosureNaive, > >> > >>> >> > >>>> TriangleEnumerationNaive/Opt, > >> > >>> >> > >>>> >> >>> PageRank > >> > >>> >> > >>>> >> >>> > >> > >>> >> > >>>> >> >>> These are the examples people called dibs on: > >> > >>> >> > >>>> >> >>> - BatchGradientDescent (M=C3=A1rton) (Should b= e a > port > >> of > >> > >>> >> > >>>> LinearRegression > >> > >>> >> > >>>> >> >>> Example from Java) > >> > >>> >> > >>>> >> >>> - ComputeEdgeDegrees (Hermann) > >> > >>> >> > >>>> >> >>> > >> > >>> >> > >>>> >> >>> Those are unclaimed (if I'm not mistaken): > >> > >>> >> > >>>> >> >>> - The relational Stuff > >> > >>> >> > >>>> >> >>> > >> > >>> >> > >>>> >> >>> On Thu, Sep 11, 2014 at 3:06 PM, Stephan Ewen < > >> > >>> >> > sewen@apache.org> > >> > >>> >> > >>>> >> wrote: > >> > >>> >> > >>>> >> >>> > +1 for removing RelationQuery > >> > >>> >> > >>>> >> >>> > > >> > >>> >> > >>>> >> >>> > On Thu, Sep 11, 2014 at 3:04 PM, Aljoscha > >> Krettek < > >> > >>> >> > >>>> >> aljoscha@apache.org> > >> > >>> >> > >>>> >> >>> > wrote: > >> > >>> >> > >>>> >> >>> > > >> > >>> >> > >>>> >> >>> >> By the way, what was called > >> BatchGradientDescent in > >> > >> the > >> > >>> >> > Scala > >> > >>> >> > >>>> >> examples > >> > >>> >> > >>>> >> >>> >> should be replaced by a port of the > >> > LinearRegression > >> > >>> >> > Example from > >> > >>> >> > >>>> >> >>> >> Java. I had them as two separate examples > >> earlier. > >> > >>> >> > >>>> >> >>> >> > >> > >>> >> > >>>> >> >>> >> What about RelationalQuery and TPC-H-Q3. Any > >> > thoughts > >> > >>> >> about > >> > >>> >> > >>>> removing > >> > >>> >> > >>>> >> >>> >> RelationalQuery? > >> > >>> >> > >>>> >> >>> >> > >> > >>> >> > >>>> >> >>> >> On Thu, Sep 11, 2014 at 11:43 AM, Aljoscha > >> Krettek > >> > < > >> > >>> >> > >>>> >> aljoscha@apache.org > >> > >>> >> > >>>> >> >>> > > >> > >>> >> > >>>> >> >>> >> wrote: > >> > >>> >> > >>>> >> >>> >> > I added the Triangle Enumeration Examples, > >> thanks > >> > >>> >> Fabian. > >> > >>> >> > >>>> >> >>> >> > > >> > >>> >> > >>>> >> >>> >> > So far we have ported: WordCount, KMeans, > >> > >>> >> > ConnectedComponents, > >> > >>> >> > >>>> >> >>> >> > WebLogAnalysis, TransitiveClosureNaive, > >> > >>> >> > >>>> >> TriangleEnumerationNaive/Opt > >> > >>> >> > >>>> >> >>> >> > > >> > >>> >> > >>>> >> >>> >> > These are the examples people called dibs > on: > >> > >>> >> > >>>> >> >>> >> > - PageRank (Fabian) > >> > >>> >> > >>>> >> >>> >> > - BatchGradientDescent (M=C3=A1rton) > >> > >>> >> > >>>> >> >>> >> > - ComputeEdgeDegrees (Hermann) > >> > >>> >> > >>>> >> >>> >> > > >> > >>> >> > >>>> >> >>> >> > Those are unclaimed (if I'm not mistaken): > >> > >>> >> > >>>> >> >>> >> > - The relational Stuff > >> > >>> >> > >>>> >> >>> >> > - LinearRegression > >> > >>> >> > >>>> >> >>> >> > > >> > >>> >> > >>>> >> >>> >> > On Wed, Sep 10, 2014 at 6:04 PM, Aljoscha > >> > Krettek < > >> > >>> >> > >>>> >> >>> aljoscha@apache.org> > >> > >>> >> > >>>> >> >>> >> wrote: > >> > >>> >> > >>>> >> >>> >> >> Thanks, I added it. I'll keep a running > list > >> of > >> > >>> >> > >>>> ported/unported > >> > >>> >> > >>>> >> >>> >> >> examples in my mails. I'll rename the jav= a > >> > example > >> > >>> >> > package to > >> > >>> >> > >>>> >> >>> examples > >> > >>> >> > >>>> >> >>> >> >> once the Scala API merge is done. > >> > >>> >> > >>>> >> >>> >> >> > >> > >>> >> > >>>> >> >>> >> >> I think the termination criterion is fine > as > >> it > >> > is. > >> > >>> >> Just > >> > >>> >> > >>>> because > >> > >>> >> > >>>> >> >>> Scala > >> > >>> >> > >>>> >> >>> >> >> enables functional programming doesn't me= an > >> it's > >> > >> always > >> > >>> >> > the > >> > >>> >> > >>>> best > >> > >>> >> > >>>> >> >>> >> >> choice. :D > >> > >>> >> > >>>> >> >>> >> >> > >> > >>> >> > >>>> >> >>> >> >> So far we have ported: WordCount, KMeans, > >> > >>> >> > ConnectedComponents, > >> > >>> >> > >>>> >> >>> >> >> WebLogAnalysis, TransitiveClosureNaive > >> > >>> >> > >>>> >> >>> >> >> > >> > >>> >> > >>>> >> >>> >> >> These are the examples people called dibs > on: > >> > >>> >> > >>>> >> >>> >> >> - TriangleEnumration and PageRank (Fabia= n) > >> > >>> >> > >>>> >> >>> >> >> - BatchGradientDescent (M=C3=A1rton) > >> > >>> >> > >>>> >> >>> >> >> - ComputeEdgeDegrees (Hermann) > >> > >>> >> > >>>> >> >>> >> >> > >> > >>> >> > >>>> >> >>> >> >> Those are unclaimed (if I'm not mistaken)= : > >> > >>> >> > >>>> >> >>> >> >> - The relational Stuff > >> > >>> >> > >>>> >> >>> >> >> - LinearRegression > >> > >>> >> > >>>> >> >>> >> >> > >> > >>> >> > >>>> >> >>> >> >> Cheers, > >> > >>> >> > >>>> >> >>> >> >> Aljoscha > >> > >>> >> > >>>> >> >>> >> >> > >> > >>> >> > >>>> >> >>> >> >> On Wed, Sep 10, 2014 at 4:23 PM, Kostas > >> Tzoumas > >> > < > >> > >>> >> > >>>> >> ktzoumas@apache.org > >> > >>> >> > >>>> >> >>> > > >> > >>> >> > >>>> >> >>> >> wrote: > >> > >>> >> > >>>> >> >>> >> >>> Transitive closure here, I also added a > >> > >> termination > >> > >>> >> > criterion > >> > >>> >> > >>>> >> in the > >> > >>> >> > >>>> >> >>> >> Java > >> > >>> >> > >>>> >> >>> >> >>> version: > >> > >>> >> > >>>> >> >>> >> > >> > >>> >> > >>>> > >> > >> https://github.com/ktzoumas/incubator-flink/tree/tc-scala-example > >> > >>> >> > >>>> >> >>> >> >>> > >> > >>> >> > >>>> >> >>> >> >>> Perhaps you can make the termination > >> criterion > >> > in > >> > >>> >> Scala > >> > >>> >> > more > >> > >>> >> > >>>> >> >>> >> functional? > >> > >>> >> > >>>> >> >>> >> >>> > >> > >>> >> > >>>> >> >>> >> >>> I noticed that the examples package name > is > >> > >>> >> > example.java but > >> > >>> >> > >>>> >> >>> >> examples.scala > >> > >>> >> > >>>> >> >>> >> >>> > >> > >>> >> > >>>> >> >>> >> >>> Kostas > >> > >>> >> > >>>> >> >>> >> >>> > >> > >>> >> > >>>> >> >>> >> >>> On Tue, Sep 9, 2014 at 6:12 PM, Kostas > >> Tzoumas > >> > < > >> > >>> >> > >>>> >> ktzoumas@apache.org > >> > >>> >> > >>>> >> >>> > > >> > >>> >> > >>>> >> >>> >> wrote: > >> > >>> >> > >>>> >> >>> >> >>>> > >> > >>> >> > >>>> >> >>> >> >>>> I'll take TransitiveClosure and > >> PiEstimation > >> > >> (was not > >> > >>> >> > on > >> > >>> >> > >>>> your > >> > >>> >> > >>>> >> >>> list). > >> > >>> >> > >>>> >> >>> >> >>>> > >> > >>> >> > >>>> >> >>> >> >>>> If nobody volunteers for the relational > >> stuff > >> > I > >> > >> can > >> > >>> >> > take > >> > >>> >> > >>>> those > >> > >>> >> > >>>> >> as > >> > >>> >> > >>>> >> >>> >> well. > >> > >>> >> > >>>> >> >>> >> >>>> > >> > >>> >> > >>>> >> >>> >> >>>> How about removing the "RelationalQuery= " > >> from > >> > >> both > >> > >>> >> > Scala and > >> > >>> >> > >>>> >> Java? > >> > >>> >> > >>>> >> >>> It > >> > >>> >> > >>>> >> >>> >> >>>> seems to be a proper subset of TPC-H Q3= . > >> Does > >> > it > >> > >> add > >> > >>> >> > some > >> > >>> >> > >>>> >> teaching > >> > >>> >> > >>>> >> >>> >> value on > >> > >>> >> > >>>> >> >>> >> >>>> top of TPC-H Q3? > >> > >>> >> > >>>> >> >>> >> >>>> > >> > >>> >> > >>>> >> >>> >> >>>> Kostas > >> > >>> >> > >>>> >> >>> >> >>>> > >> > >>> >> > >>>> >> >>> >> >>>> On Tue, Sep 9, 2014 at 5:57 PM, Aljosch= a > >> > Krettek > >> > >> < > >> > >>> >> > >>>> >> >>> aljoscha@apache.org > >> > >>> >> > >>>> >> >>> >> > > >> > >>> >> > >>>> >> >>> >> >>>> wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > >>> >> > >>>> >> >>> >> >>>>> Thanks, I added it, along with an > ITCase. > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > >>> >> > >>>> >> >>> >> >>>>> So far we have ported: WordCount, > KMeans, > >> > >>> >> > >>>> ConnectedComponents, > >> > >>> >> > >>>> >> >>> >> >>>>> WebLogAnalysis > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > >>> >> > >>>> >> >>> >> >>>>> These are the examples people called > dibs > >> on: > >> > >>> >> > >>>> >> >>> >> >>>>> - TriangleEnumration and PageRank > >> (Fabian) > >> > >>> >> > >>>> >> >>> >> >>>>> - BatchGradientDescent (M=C3=A1rton) > >> > >>> >> > >>>> >> >>> >> >>>>> - ComputeEdgeDegrees (Hermann) > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > >>> >> > >>>> >> >>> >> >>>>> Those are unclaimed (if I'm not > mistaken): > >> > >>> >> > >>>> >> >>> >> >>>>> - TransitiveClosure > >> > >>> >> > >>>> >> >>> >> >>>>> - The relational Stuff > >> > >>> >> > >>>> >> >>> >> >>>>> - LinearRegression > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > >>> >> > >>>> >> >>> >> >>>>> Cheers, > >> > >>> >> > >>>> >> >>> >> >>>>> Aljoscha > >> > >>> >> > >>>> >> >>> >> >>>>> > >> > >>> >> > >>>> >> >>> >> >>>>> On Tue, Sep 9, 2014 at 5:21 PM, Kostas > >> > Tzoumas < > >> > >>> >> > >>>> >> >>> ktzoumas@apache.org> > >> > >>> >> > >>>> >> >>> >> >>>>> wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> > WebLog here: > >> > >>> >> > >>>> >> >>> >> >>>>> > > >> > >>> >> > >>>> >> >>> >> >>>>> > > >> > >>> >> > >>>> >> >>> >> > >> > >>> >> > >>>> >> >>> > >> > >>> >> > >>>> >> > >> > >>> >> > >>>> > >> > >>> >> > > >> > >>> >> > >> > >> > >> > > >> > https://github.com/ktzoumas/incubator-flink/tree/webloganalysis-example-s= cala > >> > >>> >> > >>>> >> >>> >> >>>>> > > >> > >>> >> > >>>> >> >>> >> >>>>> > Do you need any more done? > >> > >>> >> > >>>> >> >>> >> >>>>> > > >> > >>> >> > >>>> >> >>> >> >>>>> > On Tue, Sep 9, 2014 at 3:08 PM, > Aljoscha > >> > >> Krettek < > >> > >>> >> > >>>> >> >>> >> aljoscha@apache.org> > >> > >>> >> > >>>> >> >>> >> >>>>> > wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> I added the ConnectedComponents > Example > >> > from > >> > >>> >> Vasia. > >> > >>> >> > >>>> >> >>> >> >>>>> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> Keep 'em coming, people. :D > >> > >>> >> > >>>> >> >>> >> >>>>> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> On Mon, Sep 8, 2014 at 6:07 PM, > Fabian > >> > >> Hueske < > >> > >>> >> > >>>> >> >>> fhueske@apache.org > >> > >>> >> > >>>> >> >>> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Alright, will do. > >> > >>> >> > >>>> >> >>> >> >>>>> >> > Thanks! > >> > >>> >> > >>>> >> >>> >> >>>>> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> > 2014-09-08 17:48 GMT+02:00 Aljosc= ha > >> > >> Krettek < > >> > >>> >> > >>>> >> >>> >> aljoscha@apache.org>: > >> > >>> >> > >>>> >> >>> >> >>>>> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Ok people, executive decision. := D > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Please look at KMeansData.java a= nd > >> > >>> >> > KMeans.scala. I'm > >> > >>> >> > >>>> >> storing > >> > >>> >> > >>>> >> >>> >> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> data > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> in multi-dimensional object arra= ys > >> and > >> > >> then > >> > >>> >> > >>>> converting > >> > >>> >> > >>>> >> it to > >> > >>> >> > >>>> >> >>> >> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> required Java or Scala objects. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Also, I changed isEqualTo to > >> equalTo to > >> > >> make > >> > >>> >> it > >> > >>> >> > >>>> >> consistent > >> > >>> >> > >>>> >> >>> >> with the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Java > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> API. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> Regarding Join (and coGroup). > There > >> is > >> > no > >> > >> need > >> > >>> >> > for a > >> > >>> >> > >>>> >> >>> keyword, > >> > >>> >> > >>>> >> >>> >> you > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> can > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> just write: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> left.join(right).where(0).equalTo(1) { > >> > >> (le, > >> > >>> >> re) > >> > >>> >> > =3D> > >> > >>> >> > >>>> new > >> > >>> >> > >>>> >> >>> >> MyResult(le, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> re) > >> > >>> >> > >>>> >> >>> >> >>>>> >> } > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> On Mon, Sep 8, 2014 at 2:07 PM, > >> Fabian > >> > >> Hueske > >> > >>> >> < > >> > >>> >> > >>>> >> >>> >> fhueske@apache.org> > >> > >>> >> > >>>> >> >>> >> >>>>> >> wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Aside from the DataSet issue, = I > >> also > >> > >> found > >> > >>> >> an > >> > >>> >> > >>>> >> >>> inconsistency > >> > >>> >> > >>>> >> >>> >> with > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > the > >> > >>> >> > >>>> >> >>> >> >>>>> >> Java > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > API. In Java join is done as: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > >> ds1.join(ds2).where(...).equalTo(...) > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > where in the current Scala thi= s > >> is: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > >> > ds1.join(d2).where(...).isEqualTo(...) > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > isEqualTo() should be renamed = to > >> > >> equalTo(), > >> > >>> >> > IMO. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > Also, join (+cross and coGroup= ?) > >> > lacks > >> > >> the > >> > >>> >> > with() > >> > >>> >> > >>>> >> method > >> > >>> >> > >>>> >> >>> >> because > >> > >>> >> > >>>> >> >>> >> >>>>> >> "with" > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> is > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > a keyword in Scala. Should be > >> offer > >> > >>> >> something > >> > >>> >> > >>>> similar > >> > >>> >> > >>>> >> for > >> > >>> >> > >>>> >> >>> >> Scala > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > or go > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> with > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > map() on Tuple2(left, right)? > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > 2014-09-08 13:51 GMT+02:00 > Stephan > >> > Ewen > >> > >> < > >> > >>> >> > >>>> >> sewen@apache.org > >> > >>> >> > >>>> >> >>> >: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Instead of Strings, Object[][= ] > >> would > >> > >> work > >> > >>> >> as > >> > >>> >> > well. > >> > >>> >> > >>>> >> That > >> > >>> >> > >>>> >> >>> is a > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> generic > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> representation of a Tuple. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> Alternatively, they could be > >> stored > >> > as > >> > >> Java > >> > >>> >> > or > >> > >>> >> > >>>> Scala > >> > >>> >> > >>>> >> >>> Tuples, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> with a > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> generic > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> utility method to convert > between > >> > the > >> > >> two. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> On Mon, Sep 8, 2014 at 10:55 > AM, > >> > Fabian > >> > >>> >> > Hueske > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > Yeah, I ran into the same > >> > problem... > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > +1 for using Strings and > >> parsing > >> > >> them, > >> > >>> >> but > >> > >>> >> > >>>> using > >> > >>> >> > >>>> >> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > CSVFormat > >> > >>> >> > >>>> >> >>> >> >>>>> >> won't > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> work > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > because this is based on a > >> > >>> >> FileInputFormat. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > So we would need to parse t= he > >> > Strings > >> > >>> >> > >>>> manually... > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > 2014-09-08 10:35 GMT+02:00 > >> > Aljoscha > >> > >>> >> Krettek > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > : > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Hi, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > on second thought. Maybe = we > >> > should > >> > >> just > >> > >>> >> > change > >> > >>> >> > >>>> >> all > >> > >>> >> > >>>> >> >>> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example > >> > >>> >> > >>>> >> >>> >> >>>>> >> input > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > data to strings and use C= SV > >> > input > >> > >>> >> > formats in > >> > >>> >> > >>>> all > >> > >>> >> > >>>> >> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > examples. > >> > >>> >> > >>>> >> >>> >> >>>>> >> What > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> do > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > you think? > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Cheers, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > Aljoscha > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > On Mon, Sep 8, 2014 at 7:= 46 > >> AM, > >> > >>> >> Aljoscha > >> > >>> >> > >>>> Krettek > >> > >>> >> > >>>> >> < > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> aljoscha@apache.org> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Hi, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > yes it's unfortunate th= at > >> the > >> > >> data > >> > >>> >> > types are > >> > >>> >> > >>>> >> >>> >> incompatible. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > I'm > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> afraid > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > you have to to what you > >> > proposed: > >> > >>> >> move > >> > >>> >> > the > >> > >>> >> > >>>> >> data to > >> > >>> >> > >>>> >> >>> a > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > static > >> > >>> >> > >>>> >> >>> >> >>>>> >> field > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > convert it in the > >> > >>> >> > getDefaultEdgeDataSet() > >> > >>> >> > >>>> >> method in > >> > >>> >> > >>>> >> >>> >> Scala. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > It's > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> not > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > nice, but copying would > >> > >> duplicate the > >> > >>> >> > data > >> > >>> >> > >>>> and > >> > >>> >> > >>>> >> >>> make it > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > easier > >> > >>> >> > >>>> >> >>> >> >>>>> >> for > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> it > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > to go out of sync in th= e > >> Java > >> > and > >> > >>> >> Scala > >> > >>> >> > >>>> >> versions. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > What do the others thin= k? > >> This > >> > >> will > >> > >>> >> > probably > >> > >>> >> > >>>> >> occur > >> > >>> >> > >>>> >> >>> in > >> > >>> >> > >>>> >> >>> >> all > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Cheers, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > Aljoscha > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > On Sun, Sep 7, 2014 at > >> 10:04 > >> > PM, > >> > >>> >> > Vasiliki > >> > >>> >> > >>>> >> Kalavri > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > < > vasilikikalavri@gmail.com > >> > > >> > >> wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Hey, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> I have ported the > >> Connected > >> > >>> >> Components > >> > >>> >> > >>>> >> example, > >> > >>> >> > >>>> >> >>> but > >> > >>> >> > >>>> >> >>> >> I am > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> not > >> > >>> >> > >>>> >> >>> >> >>>>> >> sure > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> how > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > to > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> reuse the example inpu= t > >> data > >> > >> from > >> > >>> >> > >>>> >> java-examples. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> In the > >> > ConnectedComponentsData > >> > >>> >> class, > >> > >>> >> > the > >> > >>> >> > >>>> >> vertices > >> > >>> >> > >>>> >> >>> >> and > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> edges > >> > >>> >> > >>>> >> >>> >> >>>>> >> data > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> are > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> produced by the method= s > >> > >>> >> > >>>> >> getDefaultVertexDataSet() > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> and > >> getDefaultEdgeDataSet(), > >> > >> which > >> > >>> >> > take > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> an > >> > >>> >> > >>>> >> org.apache.flink.api.java.ExecutionEnvironment > >> > >>> >> > >>>> >> >>> as > >> > >>> >> > >>>> >> >>> >> >>>>> >> parameter. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> One way is to provide > >> public > >> > >> static > >> > >>> >> > fields > >> > >>> >> > >>>> >> (like > >> > >>> >> > >>>> >> >>> in > >> > >>> >> > >>>> >> >>> >> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> WordCountData > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> class), but this > >> introduces a > >> > >>> >> > conversion > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> from > >> > >>> >> > >>>> org.apache.flink.api.java.tuple.Tuple2 to > >> > >>> >> > >>>> >> >>> Scala > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> tuple and > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> from > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> java.lang.Long to > >> scala.Long > >> > >> and I > >> > >>> >> > guess > >> > >>> >> > >>>> this > >> > >>> >> > >>>> >> is > >> > >>> >> > >>>> >> >>> an > >> > >>> >> > >>>> >> >>> >> >>>>> >> unnecessary > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > complexity > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> for an example (?). > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Another way is, of > >> course, to > >> > >> copy > >> > >>> >> the > >> > >>> >> > >>>> example > >> > >>> >> > >>>> >> >>> data > >> > >>> >> > >>>> >> >>> >> in > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> Scala > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > example. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Am I missing something > >> here? > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Thanks! > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> Cheers, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> V. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> On 5 September 2014 > 15:52, > >> > >> Aljoscha > >> > >>> >> > >>>> Krettek < > >> > >>> >> > >>>> >> >>> >> >>>>> >> aljoscha@apache.org > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Alright, I updated my > >> repo: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > >>> >> > >>>> >> >>> >> > >> > >>> >> > > >> https://github.com/aljoscha/incubator-flink/commits/scala-rework > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> This now has a workin= g > >> > >> WordCount > >> > >>> >> > example. > >> > >>> >> > >>>> >> It's > >> > >>> >> > >>>> >> >>> >> pretty > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> much a > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> copy > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> of > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the Java example with > >> some > >> > >> fixups > >> > >>> >> > for the > >> > >>> >> > >>>> >> syntax > >> > >>> >> > >>>> >> >>> and > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> lambda > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > functions. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> You'll also notice > that I > >> > >> added the > >> > >>> >> > >>>> >> java-examples > >> > >>> >> > >>>> >> >>> >> as a > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> dependency > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> for > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the scala-examples. I > did > >> > this > >> > >> to > >> > >>> >> > reuse > >> > >>> >> > >>>> the > >> > >>> >> > >>>> >> >>> example > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> input > >> > >>> >> > >>>> >> >>> >> >>>>> >> data. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> When you ported a > program > >> > you > >> > >> can > >> > >>> >> do > >> > >>> >> > a > >> > >>> >> > >>>> pull > >> > >>> >> > >>>> >> >>> request > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> against > >> > >>> >> > >>>> >> >>> >> >>>>> >> my > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> repo > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> and I will collect th= e > >> > >> examples. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> Happy coding. :D > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> On Fri, Sep 5, 2014 a= t > >> 12:19 > >> > >> PM, > >> > >>> >> > Hermann > >> > >>> >> > >>>> >> G=C3=A1bor < > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> reckoner42@gmail.com > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > +1 > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > ComputeEdgeDegrees > for > >> me! > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > On Fri, Sep 5, 2014 > at > >> > 11:44 > >> > >> AM, > >> > >>> >> > M=C3=A1rton > >> > >>> >> > >>>> >> >>> Balassi < > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > balassi.marton@gmail.com > >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> +1 > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> BatchGradientDesce= nt > >> for > >> > me > >> > >> :) > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> On Fri, Sep 5, 201= 4 > at > >> > >> 11:15 AM, > >> > >>> >> > Kostas > >> > >>> >> > >>>> >> >>> Tzoumas < > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > ktzoumas@apache.org> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > +1 > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > I go for > >> > WebLogAnalysis. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > My experience wi= th > >> > Scala > >> > >>> >> > consists of > >> > >>> >> > >>>> >> going > >> > >>> >> > >>>> >> >>> >> through > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > a > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> tutorial > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> so > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > this > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> will > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be a good stress > >> test > >> > >> both for > >> > >>> >> > me and > >> > >>> >> > >>>> >> the > >> > >>> >> > >>>> >> >>> new > >> > >>> >> > >>>> >> >>> >> API > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > :-) > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > On Thu, Sep 4, > 2014 > >> at > >> > >> 9:09 > >> > >>> >> PM, > >> > >>> >> > >>>> Vasiliki > >> > >>> >> > >>>> >> >>> >> Kalavri < > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >> > vasilikikalavri@gmail.com> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > +1 for having > >> other > >> > >> people > >> > >>> >> > >>>> implement > >> > >>> >> > >>>> >> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > examples! > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > Connected > >> Components > >> > and > >> > >>> >> > Kmeans for > >> > >>> >> > >>>> >> me :) > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > -V. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > On 4 September > >> 2014 > >> > >> 21:03, > >> > >>> >> > Fabian > >> > >>> >> > >>>> >> Hueske < > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> fhueske@apache.org> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > I go for > >> > >>> >> > TriangleEnumeration and > >> > >>> >> > >>>> >> >>> PageRank. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > Let's also d= o > >> the > >> > >> examples > >> > >>> >> > >>>> similar > >> > >>> >> > >>>> >> to > >> > >>> >> > >>>> >> >>> the > >> > >>> >> > >>>> >> >>> >> Java > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> examples: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - running > >> > >> out-of-the-box > >> > >>> >> > without > >> > >>> >> > >>>> >> >>> parameters > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - parameters > for > >> > >> external > >> > >>> >> > data > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > - follow a > >> similar > >> > >> code > >> > >>> >> > structure > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > 2014-09-04 > 20:56 > >> > >> GMT+02:00 > >> > >>> >> > >>>> Aljoscha > >> > >>> >> > >>>> >> >>> >> Krettek < > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > aljoscha@apache.org > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > Will do, > then > >> > >> people can > >> > >>> >> > >>>> reserve > >> > >>> >> > >>>> >> their > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > favourite > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> examples > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > here. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > On Thu, Se= p > 4, > >> > 2014 > >> > >> at > >> > >>> >> > 8:55 PM, > >> > >>> >> > >>>> >> Fabian > >> > >>> >> > >>>> >> >>> >> Hueske > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > < > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> fhueske@apache.org> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > wrote: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Hi, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I think > >> having > >> > >>> >> examples > >> > >>> >> > >>>> >> implemented > >> > >>> >> > >>>> >> >>> by > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > differen= t > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> people > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> proved to > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > be > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > valuable > in > >> the > >> > >> past. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > I'd help > >> with > >> > two > >> > >> or > >> > >>> >> > three > >> > >>> >> > >>>> >> examples. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > It might > be > >> > >> helpful if > >> > >>> >> > you'd > >> > >>> >> > >>>> >> port a > >> > >>> >> > >>>> >> >>> >> simple > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > first > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> one > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > such > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > as > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > WordCount. > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > Fabian > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > 2014-09-= 04 > >> > 18:47 > >> > >>> >> > GMT+02:00 > >> > >>> >> > >>>> >> Aljoscha > >> > >>> >> > >>>> >> >>> >> Krettek > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > < > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> aljoscha@apache.org > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> >: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Hi, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I have = a > >> > working > >> > >>> >> > rewrite of > >> > >>> >> > >>>> the > >> > >>> >> > >>>> >> >>> Scala > >> > >>> >> > >>>> >> >>> >> API > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> here: > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >> > >>> >> > >>>> >> >>> >> > >> > >>> >> > > >> https://github.com/aljoscha/incubator-flink/commits/scala-rework > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> I'm > hoping > >> > that > >> > >> I'll > >> > >>> >> > only > >> > >>> >> > >>>> have > >> > >>> >> > >>>> >> to > >> > >>> >> > >>>> >> >>> >> write > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> tests > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> and > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > port > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> example= s. > >> Do > >> > you > >> > >>> >> think > >> > >>> >> > it > >> > >>> >> > >>>> makes > >> > >>> >> > >>>> >> >>> sense > >> > >>> >> > >>>> >> >>> >> to > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> let > >> > >>> >> > >>>> >> >>> >> >>>>> >> other > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > people > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> port > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> the > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> example= s, > >> so > >> > that > >> > >>> >> > someone > >> > >>> >> > >>>> else > >> > >>> >> > >>>> >> uses > >> > >>> >> > >>>> >> >>> >> it and > >> > >>> >> > >>>> >> >>> >> >>>>> >> maybe > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > notices > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > some > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > quirks > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> in the > API? > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Cheers, > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> Aljosch= a > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >>> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> >> > >> > >>> >> > >>>> >> >>> >> >>>>> >> > >> > >>> >> > >>>> >> >>> >> >>>> > >> > >>> >> > >>>> >> >>> >> >>>> > >> > >>> >> > >>>> >> >>> >> >>> > >> > >>> >> > >>>> >> >>> >> > >> > >>> >> > >>>> >> >>> > >> > >>> >> > >>>> >> > >> > >>> >> > >>>> > >> > >>> >> > > >> > >>> >> > >> > > >> > > > > > --047d7bf0c37e176d47050302f38f--