From user-return-24326-archive-asf-public=cust-asf.ponee.io@flink.apache.org Wed Nov 14 07:17:58 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id EE21818066B for ; Wed, 14 Nov 2018 07:17:57 +0100 (CET) Received: (qmail 19740 invoked by uid 500); 14 Nov 2018 06:17:56 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 19725 invoked by uid 99); 14 Nov 2018 06:17:56 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 14 Nov 2018 06:17:56 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 2DFD8C233A for ; Wed, 14 Nov 2018 06:17:56 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.34 X-Spam-Level: X-Spam-Status: No, score=0.34 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-1.459, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id LIL2Q4LB1QwF for ; Wed, 14 Nov 2018 06:17:54 +0000 (UTC) Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com [209.85.208.173]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 836B0623DF for ; Wed, 14 Nov 2018 06:17:54 +0000 (UTC) Received: by mail-lj1-f173.google.com with SMTP id z80-v6so13065015ljb.8 for ; Tue, 13 Nov 2018 22:17:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=CP+L3wSTKOVI9e/dWLIReWeNAl66XlEtbrIQvyEeRGY=; b=SBbwxfMIxsF8ifNa+yTvw/fTw8dH8r/jHcCZ1JKM3mxUTaQOO+n3RrSkTcMMiDp9XW qV9NT+AJjRoJuJo0jsUVTkDTp037cKFO0ez5Ts6hANW8RaSrpru/rwVZGIrhxvJf6cjV XLx9/0Tf0+jSacNpfmM9wkFp7mCcl6an58j0OUI0WDE/Ndn3ST/CFf9vIZb79zP9nkQL AgiGaY3S8yQbKKN4YRdRT1CuqhD4KsxMEqB6ZiyE8jjNTFfqTNoHOAsw3rhfRSug3rxv qy7SIfPlw7gIZdIXNqIEshQ9q0L0WaFPeNBiNyj/oSviMYOoVZ6c05SexshzJ/9MJlBF O/Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=CP+L3wSTKOVI9e/dWLIReWeNAl66XlEtbrIQvyEeRGY=; b=szzDKHrloA1f4jVn/N+dBL0RoE7Mff3IYi+378HN9S2Icdm+1Kz6fcJWkYGuHqkeQD Qw4BYfIm1sJsDl4AQUEeSKiPKoRoV1DGWXVZRMvjPhSDQNXpimU17iR+q00jHd3yUu40 N/ZqpCRIxABGnIMmXouTXxn25sAp6RUZVuJh+NJ5OTjUj8qQ9z5jD5edGDE/TRWEvfjZ mmKZ+pNmg4Tvhp46BM3LWwt/T/ynrAdkqzyngceKVhe12WlkBxJFgXzRJ3yO8yoOTxhf ZaTtLnchylF7bukg7/SEb3n31S0ngWElxOSBVqearH4B11l9clswng7AQymAr8jaeUJU ND/A== X-Gm-Message-State: AGRZ1gLF9RTm8awTAYWZ0u+dHvSFNE8VLerMYohVJnP66/WfT0AIShq/ ALc5LO5IVJ5VDrn99tY1kr3SPvPbTDZgzkOFAKI= X-Google-Smtp-Source: AJdET5epUzGJM1F6pYWkQaiB6XiWkNkd+ebmkuxQhVh5K3o64cnml/plx5U42z5xqey4x7ZpjbVwGrYI/yih3dCYZs0= X-Received: by 2002:a2e:2019:: with SMTP id g25-v6mr326512ljg.20.1542176273014; Tue, 13 Nov 2018 22:17:53 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Hequn Cheng Date: Wed, 14 Nov 2018 14:17:11 +0800 Message-ID: Subject: Re: Confused window operation To: zjffdu@gmail.com Cc: user Content-Type: multipart/alternative; boundary="000000000000ef2760057a99e4d3" --000000000000ef2760057a99e4d3 Content-Type: text/plain; charset="UTF-8" Hi Jeff, The window is not a global window. It is related to a specified key. You would have 6 windows after flatMap() and keyBy(). key: hello with 3 windows key: world with 1 window key: flink with 1 window key: hadoop with 1 window Best, Hequn On Wed, Nov 14, 2018 at 10:31 AM Jeff Zhang wrote: > Hi all, > > I am a little confused with the following windows operation. Here's the > code, > > val senv = StreamExecutionEnvironment.getExecutionEnvironment > senv.setParallelism(1) > val data = senv.fromElements("hello world", "hello flink", "hello hadoop") > > data.flatMap(line => line.split("\\s")) > .map(w => (w, 1)) > .keyBy(0) > .countWindow(2, 1) > .sum(1) > .print("******************") > > senv.execute() > > > And this is the output: > > ******************> (hello,1) > ******************> (world,1) > ******************> (hello,2) > ******************> (flink,1) > ******************> (hello,2) > ******************> (hadoop,1) > > > As my understanding, here we have 3 windows. > > window 1 > > (hello, world) > > window 2 > > (hello, world) > > (hello, flink) > > window 3 > > (hello flink) > > (hello hadoop) > > So for the first window, we have output (hello, 1) (world, 1) > > for the second window we should output (hello, 2), (world,1 ), (flink, 1) > > for the third window we should have output (hello, 2), (flink, 1), (hadoop, 1) > > > But as you can see, in the above I get different result, do I misunderstand the window ? Could anyone help me to understand that ? Thanks > > --000000000000ef2760057a99e4d3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Jeff,

<= div>The window is not a global window. It is related to a specified key. Yo= u would have 6 windows after flatMap() and keyBy().
key: hello wi= th 3 windows
key: world with 1 window
key: flink=C2=A0w= ith 1 window
key: hadoop=C2=A0with 1 window

<= div>Best, Hequn


On Wed, Nov 14, 2018 at 10:31 AM Jeff Zhang <= zjffdu@gmail.com> wrote:
Hi all,

I am a little confused with the following windows operation. Here's t= he code,

val senv =
=3D StreamExecutionEnvironment.getExecuti=
onEnvironment
senv.setParallelism(1)
val data =3D senv.fromElements("hello world", "hello flink", "hello hadoop")

data.fla=
tMap(line =3D> line.split("\\s"))  .map(w =3D> (w, 1))
  .=
keyBy(0)
  .countWindow(2, 1=
)
  .sum(1)
  .print=
("****************=
**")

senv.execute()

An=
d this is the output:
******************>=
 (hello,1)
******************> (world,1)
******************> (hello,2)
******************> (flink,1)
******************> (hello,2)
******************> (hadoop,1)
<=
/font>

As my understanding, here we have 3 windows.
window 1
(hello, world)=

window 2
(hello, world)
(hello, flink=
)
window 3
(hello flink)
(hello hadoop)
= 
So for the first window, we have output (hello, 1) (world, 1)
for the second window we should output (hello, 2), (world,1 ), (flink, 1)= 
for the third window we should have output (hello, 2), (flink, 1=
), (hadoop, 1)

But as you can see, in the above I =
get different result, do I misunderstand the window ? Could anyone help me =
to understand that ? Thanks
--000000000000ef2760057a99e4d3--