Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 7C568200B43 for ; Tue, 19 Jul 2016 15:09:03 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 7AF86160A8B; Tue, 19 Jul 2016 13:09:03 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 9BF56160A89 for ; Tue, 19 Jul 2016 15:09:02 +0200 (CEST) Received: (qmail 4488 invoked by uid 500); 19 Jul 2016 13:09:01 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 4478 invoked by uid 99); 19 Jul 2016 13:09:01 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Jul 2016 13:09:01 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 301EDC0958 for ; Tue, 19 Jul 2016 13:09:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.179 X-Spam-Level: * X-Spam-Status: No, score=1.179 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id R6tCoLuCNs22 for ; Tue, 19 Jul 2016 13:08:59 +0000 (UTC) Received: from mail-qk0-f177.google.com (mail-qk0-f177.google.com [209.85.220.177]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id C69015F5A4 for ; Tue, 19 Jul 2016 13:08:58 +0000 (UTC) Received: by mail-qk0-f177.google.com with SMTP id x1so14737395qkb.3 for ; Tue, 19 Jul 2016 06:08:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=T82itIH5n3bseePLWVHX6+0dZYoPEOVSxd4wuMGkLM4=; b=P5HFFj0aqFAu1tJ3ixZ9zjvBhX4Fo+rTQTjiPWSG/QuJT2xAanCA0n0C7G0qoDFTVa MWLCmDBAdmblYuuOzHs/5uZR1EwYXaBH+N8q+vteHAughfLRCOjyBohJ6DFr91HtlO5l +5geumNYnMz492QNa0SnK/sV9sDkad9B3dmj7BpA36VDSM02ssRJx+sLrhbGENeuMGeR kDke9FOpNkvoa94onSUDyjApHpR4MAKCA4Sq8tDikFsD7e2FIY24LX3e0gJ53krg83q4 acMTyTJszBgc6YfGv6S2jOmd6qU+uTQ5P2BaiFRy7NvyC2494GUMKC08eKPjgCrgSs4U CEqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=T82itIH5n3bseePLWVHX6+0dZYoPEOVSxd4wuMGkLM4=; b=TUpfE8LPpleGGNexm7YDXrkllki6zomzaGM73WJpNUXTlqQqg5COyJURyQgj82YXpD A+kop7kbhq1aDkfaQ9/L51DCtNb5LROdL0WYiBRR1uCPeo1zHIpAz60w5NP7ZGmUyddO yDLhxm7tnI8v4iIZKBvy0CQsqoYohDUaDsnQCopbwu86kUb38sTzH8iA/OaZEARCTJ4O BbRSfZsHviuV121pHiIwDg5UuuBo0RjcrhslupBMASrrO9cr36ZRBKSAgRkgTYWU6Sv5 gPNTmrqMW93+9HPws02948+4FhiG6xR8VPWTaWJdI5roo6RVwTZQCKeCx9ufJgt3duYj A42A== X-Gm-Message-State: ALyK8tIrEpoKFeYBCpnjqsW+jN6ukNRZ2VX1cQOfQRvYcum8hqNkhQjsdkxiXoyQ0Ui0szMmtsmh2HXIcY8B7A== X-Received: by 10.55.126.7 with SMTP id z7mr50157874qkc.172.1468933731736; Tue, 19 Jul 2016 06:08:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.55.78.79 with HTTP; Tue, 19 Jul 2016 06:08:51 -0700 (PDT) In-Reply-To: References: From: subash basnet Date: Tue, 19 Jul 2016 15:08:51 +0200 Message-ID: Subject: Re: DataStreamUtils not working properly To: user@flink.apache.org Content-Type: multipart/alternative; boundary=94eb2c063b7a4760b70537fcc90c archived-at: Tue, 19 Jul 2016 13:09:03 -0000 --94eb2c063b7a4760b70537fcc90c Content-Type: text/plain; charset=UTF-8 Hello Till, Shouldn't it write something in the eclipse console if there is any error or warning. But nothing about error is printed on the console. And I checked the flink project folder: flink-core, flink streaming as such but couldn't find where the log is written when run via eclipse. Best Regards, Subash Basnet On Tue, Jul 19, 2016 at 2:49 PM, Till Rohrmann wrote: > Have you checked your logs whether they contain some problems? In general > it is not recommended collecting the streaming result back to your client. > It might also be a problem with `DataStreamUtils.collect`. > > Cheers, > Till > > On Tue, Jul 19, 2016 at 2:42 PM, subash basnet wrote: > >> Hello all, >> >> I tried to check if it works for tuple but same problem, the collection >> still shows blank result. I took the id of centroid tuple and printed it, >> but the collection displays empty. >> >> DataStream centroids = newCentroidDataStream.map(new >> TupleCentroidConverter()); >> DataStream> centroidId = centroids.map(new TestMethod()); >> centroidId.print(); >> Iterator> iter = DataStreamUtils.collect(centroidId); >> Collection> testCentroids = Lists.newArrayList(iter); >> for (Tuple1 c : testCentroids) { >> System.out.println(c); >> } >> Output: (Mon Jul 18 17:36:03 CEST 2016) (Mon Jul 18 17:43:58 CEST 2016) >> (Mon Jul 18 17:42:59 CEST 2016) (Mon Jul 18 17:34:01 CEST 2016) (Mon Jul 18 >> 17:52:00 CEST 2016) (Mon Jul 18 17:40:58 CEST 2016) for centroidId.print(), >> but no output for System.out.println(c); Best Regards, Subash Basnet >> >> On Tue, Jul 19, 2016 at 10:48 AM, subash basnet >> wrote: >> >>> Hello all, >>> >>> I am trying to convert datastream to collection, but it's shows blank >>> result. There is a stream of data which can be viewed on the console on >>> print(), but the collection of the same stream shows empty after >>> conversion. Below is the code: >>> >>> DataStream centroids = newCentroidDataStream.map(new >>> TupleCentroidConverter()); >>> centroids.print(); >>> Iterator iter = DataStreamUtils.collect(centroids); >>> Collection testCentroids = Lists.newArrayList(iter); >>> for(Centroid c: testCentroids){ >>> System.out.println(c); >>> } >>> >>> The above *centroids.print()* gives the following output in console: >>> >>> Mon Jul 18 21:29:01 CEST 2016 119.3701 119.4 119.3701 119.38 27400.0 >>> Mon Jul 18 21:23:00 CEST 2016 119.3463 119.37 119.315 119.37 48200.0 >>> Mon Jul 18 21:27:59 CEST 2016 119.3401 119.3401 119.26 119.265 50300.0 >>> Mon Jul 18 21:36:00 CEST 2016 119.48 119.505 119.47 119.4741 37400.0 >>> Mon Jul 18 21:33:00 CEST 2016 119.535 119.54 119.445 119.455 152900.0 >>> >>> But the next *System.out.println(c) *within the for loop prints >>> nothing. What could be the problem. >>> >>> My maven has following configuration for dataStreamUtils: >>> >>> org.apache.flink >>> flink-streaming-contrib_2.10 >>> ${flink.version} >>> >>> >>> >>> Best Regards, >>> Subash Basnet >>> >>> >> > --94eb2c063b7a4760b70537fcc90c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hello Till,

Shouldn't it write something i= n the eclipse console if there is any error or warning. But nothing about e= rror is printed on the console. And I checked the flink project folder: fli= nk-core, flink streaming as such but couldn't find where the log is wri= tten when run via eclipse.=C2=A0

Best Regards,
Subash Basnet

On Tue, Ju= l 19, 2016 at 2:49 PM, Till Rohrmann <trohrmann@apache.org> wrote:
Have you check= ed your logs whether they contain some problems? In general it is not recom= mended collecting the streaming result back to your client. It might also b= e a problem with `DataStreamUtils.collect`.

Cheers,
Till

On Tue, Jul 19, 2016 at 2:42 P= M, subash basnet <yasubash@gmail.com> wrote:
Hello all,

I tried to check if= it works for tuple but same problem, the collection still shows blank resu= lt. I took the id of centroid tuple and printed it, but the collection disp= lays empty.

DataStream<Centroid> centroids =3D newC= entroidDataStream.map(new TupleCentroidConverter());
DataStream<Tuple1<String>= > centroidId =3D centroids.map(new TestMethod());
centroidId.print();
Iterator<Tuple1<String>> it= er =3D DataStreamUtils.collect(centroidId);
Collection<Tuple1<String>> testCentroi= ds =3D Lists.newArrayList(iter);
for (Tuple1<String> c : testCentroids) {
System.out.println(c);
}
Output:=20 (Mon Jul 18 17:36:03 CEST 2016) (Mon Jul 18 17:43:58 CEST 2016) (Mon Jul 18 17:42:59 CEST 2016) (Mon Jul 18 17:34:01 CEST 2016) (Mon Jul 18 17:52:00 CEST 2016) (Mon Jul 18 17:40:58 CEST 2016) for centroidId.print(), but no output for System.out.println(c); Best Regards, Subash Basnet

On Tue, Jul 19, 2016 at 10:48 AM, subash basnet <yasubash@gmail.com> wrote:
Hello all,

I am trying to convert datastream to co= llection, but it's shows blank result. There is a stream of data which = can be viewed on the console on print(), but the collection of the same str= eam shows empty after conversion. Below is the code:

DataStream<Centroid> centroids =3D newCentroidDataStream.map(new Tu= pleCentroidConverter());
<= /span>centroids.print();
Iterator<Centroid> iter =3D DataSt= reamUtils.collect(centroids);
Collection<Centroid> testCent= roids =3D Lists.newArrayList(iter);
for(Centroid c: testCentroids= ){
System.out.printl= n(c);
}

The above centroids.print()=C2=A0gives the = following output in console:

Mon Jul 18 21:29:= 01 CEST 2016 =C2=A0119.3701 119.4 119.3701 119.38 27400.0
Mon Jul= 18 21:23:00 CEST 2016 =C2=A0119.3463 119.37 119.315 119.37 48200.0
Mon Jul 18 21:27:59 CEST 2016 =C2=A0119.3401 119.3401 119.26 119.265 503= 00.0
Mon Jul 18 21:36:00 CEST 2016 =C2=A0119.48 119.505 119.47 11= 9.4741 37400.0
Mon Jul 18 21:33:00 CEST 2016 =C2=A0119.535 119.54= 119.445 119.455 152900.0

But the next System.out.println(c) = within the for loop prints nothing. What could be the problem.=C2=A0
My maven has following configuration for dataStreamUtils:
<depe= ndency>
<group= Id>org.apache.flink</groupId>
<artifactId>flink-streaming-contrib_2.10</artif= actId>
<versio= n>${flink.version}</version>
</dependency>

=
Best Regards,
Subash Basnet




--94eb2c063b7a4760b70537fcc90c--