From user-return-22478-archive-asf-public=cust-asf.ponee.io@flink.apache.org Wed Aug 29 12:25:20 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id AD2BE180657 for ; Wed, 29 Aug 2018 12:25:19 +0200 (CEST) Received: (qmail 56959 invoked by uid 500); 29 Aug 2018 10:25:18 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 56949 invoked by uid 99); 29 Aug 2018 10:25:18 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Aug 2018 10:25:18 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 0C5ED180A7D for ; Wed, 29 Aug 2018 10:25:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.701 X-Spam-Level: *** X-Spam-Status: No, score=3.701 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FORGED_HOTMAIL_RCVD2=1.187, HK_RANDOM_ENVFROM=0.626, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=hotmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id NQMSn3DzD5Ac for ; Wed, 29 Aug 2018 10:25:15 +0000 (UTC) Received: from APC01-PU1-obe.outbound.protection.outlook.com (mail-oln040092254081.outbound.protection.outlook.com [40.92.254.81]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 0EA565F180 for ; Wed, 29 Aug 2018 10:25:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ctjoC/awH88Be3aAdfrOjEWWsr3qflyjb7W0aZfXP44=; b=ASkIUXlghbO+1M7PPq6XzE46LzPnBf8ShTzFvSsd4vqw9q0+Df01WvHdSQElaQY5H15rb7kNnrNZQpY4nGm2up81MVwwCzTbbbs24zFpsAQgvQ3wArF+seGXU+ooiU03CmSAvEMV9PWq9w/eGGDLJppanC303Zc+vXRR0enGfoeOYvcAReAKSuZoQZTpQ+hXLU9J20I6xLTdLm0SQL4a3RvkSftXXV4jXGBZNmJ53A+kiCMSJc8yo0r88md+6CdqapRjfdmlZ5zK8MVpWJAGEhXaw+2GpYnLIINIObopP9QiWcj3IG3MKmIikldudGyfqLcz9ru0CCkO+m5IwglNtQ== Received: from PU1APC01FT115.eop-APC01.prod.protection.outlook.com (10.152.252.51) by PU1APC01HT135.eop-APC01.prod.protection.outlook.com (10.152.252.197) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.1101.10; Wed, 29 Aug 2018 10:25:03 +0000 Received: from PS1PR0401MB2186.apcprd04.prod.outlook.com (10.152.252.51) by PU1APC01FT115.mail.protection.outlook.com (10.152.252.208) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.20.1101.10 via Frontend Transport; Wed, 29 Aug 2018 10:25:03 +0000 Received: from PS1PR0401MB2186.apcprd04.prod.outlook.com ([fe80::ddba:cf5b:769e:8391]) by PS1PR0401MB2186.apcprd04.prod.outlook.com ([fe80::ddba:cf5b:769e:8391%9]) with mapi id 15.20.1101.007; Wed, 29 Aug 2018 10:25:03 +0000 From: zhen li To: "user@flink.apache.org" Subject: share the big data structure between slots Thread-Topic: share the big data structure between slots Thread-Index: AQHUP4KMphJl868rVky9JPTMyE+8xQ== Date: Wed, 29 Aug 2018 10:25:03 +0000 Message-ID: Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: HK0P153CA0033.APCP153.PROD.OUTLOOK.COM (2603:1096:203:17::21) To PS1PR0401MB2186.apcprd04.prod.outlook.com (2603:1096:803:4::18) x-incomingtopheadermarker: OriginalChecksum:275C25B464A32C48FB17C9896851B4C94F47C61970EB02026CD5FCDBC54096C2;UpperCasedChecksum:E400FAC91A4A1A402D452FC9BC7109D0A971B9E307F78B97C99E11D4CACA9998;SizeAsReceived:7285;Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [K08U0Vqm24+2TX2SdhzhG43CRMfKI0rC] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;PU1APC01HT135;6:DHtQGDFyH+7Qdv3c5mbJCG7tiLivFGB4BL5XcyH1v1ZxM0Oa4O6ERZrgRoqr2mZuQk5v3X6M+35QDIUc1hMIfjAkYcdIKj5qne6roFK/nVPSNIGm9denY/wevfoLNSBaYkyf2xiWdh4JPC99MuwDH36ZELvjReK2nTx1M2vuxzW5VoK3h+d0OFHLVgRro/50T1zD7vR+00QQ2Tto9+mTjHXLxfl1+U8ldaS2v28meXVGojq0XY/Rx2YTcF74JcaW9N/Qc0hR6+f3tez6hE7D86/iJhsFcZ5FA9PUj4670WY38+hLfA+ikSAkua1tOGhjB+0LD3J7T0acM5bYkHB4UjAOOIV7raROmo0R6tTTs6TrSw+koQzNbjGwUjsKOcv+DmH0jbYLU4fP8Fh88LQtp6eMYtTT3W8QlLG/xuGK2rqyW4p2y1i9Q+A7x/IVi80VPaxCcmA49Hbw2WZXMUCI5A==;5:ywKgopQJ/wElt7RpAnazVPc2fnVC6fCYce70Qs4K1u8O8//NPtI4hfr3sSAC00NJo/LHZEOgLuc/sZrIvQl3+bmumx50nSXaYol3k7ZiWGDZCJTUcmmD6H1rTsTfC3XNS3SNCAc7GqvU0nDn902MOpU1zJD7wNbOrHmDhbYOiF4=;7:ayRlIGMboqeNJtr2Ze6oyujZ0Sd+QiRsy2qzIyXCyjpjOhEkk2IIOugjHnC+kj6DktXyf00LHYV4xeo1AwOF1Xw0Ew8JCb4vMjwVz/mSAGxk7BiRqX+hbcvin0a3xCtXTV799cjsTulXUuvVSrYwRF040XTavprllgugFevC+JiJsAFPZvKKDQstd4x2vmL0R2nWtjFW+YTqk6divWCl6Da/8Xe2OJgjDKEXGEo/lBsnjS866QwgQW0/hg0xXva5 x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(7020095)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1603101475)(1601125500)(1701031045);SRVR:PU1APC01HT135; x-ms-traffictypediagnostic: PU1APC01HT135: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(4566010)(82015058);SRVR:PU1APC01HT135;BCL:0;PCL:0;RULEID:;SRVR:PU1APC01HT135; x-forefront-prvs: 077929D941 x-forefront-antispam-report: SFV:NSPM;SFS:(7070007)(189003)(199004)(53754006)(5250100002)(6436002)(7696005)(2501003)(56003)(2351001)(20460500001)(1730700003)(9686003)(5640700003)(105586002)(106356001)(81156014)(8936002)(97736004)(74316002)(8676002)(256004)(486006)(87572001)(86362001)(14454004)(104016004)(102836004)(26005)(386003)(6506007)(6346003)(25786009)(476003)(33656002)(73972006)(82202002)(55016002)(54896002)(86152003)(6916009)(68736007)(99286004)(2900100001)(16234385003)(83332001)(5660300001)(15852004)(42522002);DIR:OUT;SFP:1901;SCL:1;SRVR:PU1APC01HT135;H:PS1PR0401MB2186.apcprd04.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: hotmail.com does not designate permitted sender hosts) authentication-results: spf=none (sender IP is ) smtp.mailfrom=lizhenmxcz@hotmail.com; x-microsoft-antispam-message-info: i+fOA5EKyjW8YN3q/ZEChzhT8cHXZNWw8f0PPLLTZGr4isHufozMpZxgaueW48+PFAEkA6v5dk3mw559di+6sjvzpA0z+A5vkK21NwswZGnZEda09QnzokMnmETK3keNDb+t3penN+++0k1cAmT8ICmivnEgbTz7I1WOOpaI0A4qi4v5a0aQT4Nrbv1H151ABSwikWmPXNEkkBC44bzzZ6UudjltZv6+MqtfbC9QM1c= Content-Type: multipart/alternative; boundary="_000_PS1PR0401MB2186772AAE6225EB44B6A923DF090PS1PR0401MB2186_" MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: f12efbb0-867f-4c93-8261-502eceebfafa X-MS-Exchange-CrossTenant-Network-Message-Id: 44eaa908-5fe3-44a1-c39a-08d60d99ada9 X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: f12efbb0-867f-4c93-8261-502eceebfafa X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Aug 2018 10:25:03.1121 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT135 --_000_PS1PR0401MB2186772AAE6225EB44B6A923DF090PS1PR0401MB2186_ Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: base64 SGkgYWxsOg0KICAgIFdoZW4gSSBicm9hZGNhc3QgdGhlIGJpZyBjb25maWcgc3RyZWFtLGV2ZXJ5 IHBhcmFsbGVsIGluc3RhbmNlIHNob3VsZCBzdG9yZSB0aGUgZGF0YSwgY2F1c2Ugd2FzdGUgb2Yg dGhlIG1vbWVyeSBhbmQgdGltZS4gSXMgdGhlcmUgc29tZSBtZXRob2QgdG8gbWFrZSB0aGUgc2xv dHMgdG8gc2hhcmUgdGhlIGJpZyBkYXRhIHN0cnVjdHVyZSB3aXRoaW4gdGhlIHRhc2ttYW5hZ2Vy o78NCkhlcmUgaXMgdGhlIGRvYzoNCiAgIEJ5IGFkanVzdGluZyB0aGUgbnVtYmVyIG9mIHRhc2sg c2xvdHMsIHVzZXJzIGNhbiBkZWZpbmUgaG93IHN1YnRhc2tzIGFyZSBpc29sYXRlZCBmcm9tIGVh Y2ggb3RoZXIuIEhhdmluZyBvbmUgc2xvdCBwZXIgVGFza01hbmFnZXIgbWVhbnMgZWFjaCB0YXNr IGdyb3VwIHJ1bnMgaW4gYSBzZXBhcmF0ZSBKVk0gKHdoaWNoIGNhbiBiZSBzdGFydGVkIGluIGEg c2VwYXJhdGUgY29udGFpbmVyLCBmb3IgZXhhbXBsZSkuIEhhdmluZyBtdWx0aXBsZSBzbG90cyBt ZWFucyBtb3JlIHN1YnRhc2tzIHNoYXJlIHRoZSBzYW1lIEpWTS4gVGFza3MgaW4gdGhlIHNhbWUg SlZNIHNoYXJlIFRDUCBjb25uZWN0aW9ucyAodmlhIG11bHRpcGxleGluZykgYW5kIGhlYXJ0YmVh dCBtZXNzYWdlcy4gVGhleSBtYXkgYWxzbyBzaGFyZSBkYXRhIHNldHMgYW5kIGRhdGEgc3RydWN0 dXJlcywgdGh1cyByZWR1Y2luZyB0aGUgcGVyLXRhc2sgb3ZlcmhlYWQuDQo= --_000_PS1PR0401MB2186772AAE6225EB44B6A923DF090PS1PR0401MB2186_ Content-Type: text/html; charset="gb2312" Content-ID: <08BCD9566264CB4AA3AC6DFC1CCEE75E@apcprd04.prod.outlook.com> Content-Transfer-Encoding: quoted-printable Hi all:
    When I broadcast the big config stream,every parallel instance should store th= e data, cause waste of the momery and time. Is there some method to make th= e slots to share the big data structure within the taskmanager=A3=BF
Here is the doc:
   By adjusting the number of task slots, users can define how subtasks are isolated from each other. Having one slot per TaskManager= means each task group runs in a separate JVM (which can be started in a se= parate container, for example). Having multiple slots means more subtasks s= hare the same JVM. Tasks in the same JVM share TCP connections (via multiplexing) and heartbeat messages. = They may also share data sets and data structures, thus reducing the per-ta= sk overhead.
--_000_PS1PR0401MB2186772AAE6225EB44B6A923DF090PS1PR0401MB2186_--