Return-Path: X-Original-To: apmail-spark-user-archive@minotaur.apache.org Delivered-To: apmail-spark-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1898118797 for ; Mon, 22 Jun 2015 21:44:07 +0000 (UTC) Received: (qmail 35711 invoked by uid 500); 22 Jun 2015 21:44:03 -0000 Delivered-To: apmail-spark-user-archive@spark.apache.org Received: (qmail 35626 invoked by uid 500); 22 Jun 2015 21:44:03 -0000 Mailing-List: contact user-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@spark.apache.org Received: (qmail 35616 invoked by uid 99); 22 Jun 2015 21:44:03 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 22 Jun 2015 21:44:03 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of silvio.fiorito@granturing.com designates 157.56.110.130 as permitted sender) Received: from [157.56.110.130] (HELO na01-bn1-obe.outbound.protection.outlook.com) (157.56.110.130) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 22 Jun 2015 21:41:46 +0000 Received: from SN2PR07MB093.namprd07.prod.outlook.com (10.255.174.157) by SN2PR07MB095.namprd07.prod.outlook.com (10.255.174.159) with Microsoft SMTP Server (TLS) id 15.1.190.14; Mon, 22 Jun 2015 21:43:32 +0000 Received: from SN2PR07MB093.namprd07.prod.outlook.com ([169.254.10.108]) by SN2PR07MB093.namprd07.prod.outlook.com ([169.254.10.108]) with mapi id 15.01.0190.013; Mon, 22 Jun 2015 21:43:32 +0000 From: Silvio Fiorito To: Jianguo Li , "user@spark.apache.org" Subject: Re: workaround for groupByKey Thread-Topic: workaround for groupByKey Thread-Index: AQHQrTAsM5Z3qlNzbkibjXKyFBFYRZ24y4YA Date: Mon, 22 Jun 2015 21:43:32 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: gmail.com; dkim=none (message not signed) header.d=none; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [108.51.146.59] x-microsoft-exchange-diagnostics: 1;SN2PR07MB095;3:JXlVqhtiuY3DNkBNe0S6V42CkvUlxTNnW3vpQJ3yqR5j67/qHN8KbDkZuAi63Nq1euT/O4lMv2jieZ+bC/1IV6hP4ojh+bK6IfAIwVAj4aodtCDAzZwMdoeK9IAKytU0ZZW/ihj2mf91AgfWByFkKg==;10:aIoF91GYuzeDj3g5nR1n2e+ccdn0XOvAdQzQq5Qzs5WPg+6gE/98qM3AEwv7akA6rUcS+wOd4u9D4xmvLFN1jtrMGJ1pFXyfDuzLHLpR2kU=;6:6QIWi+W5uQYElG1Gs7U77npQinHMAtC2MjDWrkw2Hz6HI3v8ghQIaCj1+U+3hioh x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:SN2PR07MB095; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(601004)(520004)(5005006)(3002001);SRVR:SN2PR07MB095;BCL:0;PCL:0;RULEID:;SRVR:SN2PR07MB095; x-forefront-prvs: 06157D541C x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6029001)(164054003)(377454003)(5002640100001)(2501003)(66066001)(107886002)(5001960100002)(82746002)(5001770100001)(83716003)(54356999)(62966003)(106116001)(77156002)(50986999)(76176999)(87936001)(92566002)(33656002)(2900100001)(2950100001)(2656002)(102836002)(36756003)(189998001)(16236675004)(86362001)(122556002)(19580405001)(46102003)(99286002)(40100003)(19580395003)(104396002);DIR:OUT;SFP:1102;SCL:1;SRVR:SN2PR07MB095;H:SN2PR07MB093.namprd07.prod.outlook.com;FPR:;SPF:None;MLV:sfv;LANG:en; Content-Type: multipart/alternative; boundary="_000_E34E3B513E144935A2402617023423ACgranturingcom_" MIME-Version: 1.0 X-OriginatorOrg: granturing.com X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Jun 2015 21:43:32.2145 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 80ca1563-f0f5-4035-b58f-c37cb205fb47 X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN2PR07MB095 X-Virus-Checked: Checked by ClamAV on apache.org --_000_E34E3B513E144935A2402617023423ACgranturingcom_ Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 WW91IGNhbiB1c2UgYWdncmVnYXRlQnlLZXkgYXMgb25lIG9wdGlvbjoNCg0KdmFsIGlucHV0OiBS RERbSW50LCBTdHJpbmddID0gLi4uDQoNCnZhbCB0ZXN0ID0gaW5wdXQuYWdncmVnYXRlQnlLZXko TGlzdEJ1ZmZlci5lbXB0eVtTdHJpbmddKSgoYSwgYikgPT4gYSArPSBiLCAoYSwgYikgPT4gYSAr KyBiKQ0KDQpGcm9tOiBKaWFuZ3VvIExpDQpEYXRlOiBNb25kYXksIEp1bmUgMjIsIDIwMTUgYXQg NToxMiBQTQ0KVG86ICJ1c2VyQHNwYXJrLmFwYWNoZS5vcmc8bWFpbHRvOnVzZXJAc3BhcmsuYXBh Y2hlLm9yZz4iDQpTdWJqZWN0OiB3b3JrYXJvdW5kIGZvciBncm91cEJ5S2V5DQoNCkhpLA0KDQpJ IGFtIHByb2Nlc3NpbmcgYW4gUkREIG9mIGtleS12YWx1ZSBwYWlycy4gVGhlIGtleSBpcyBhbiB1 c2VyX2lkLCBhbmQgdGhlIHZhbHVlIGlzIGFuIHdlYnNpdGUgdXJsIHRoZSB1c2VyIGhhcyBldmVy IHZpc2l0ZWQuDQoNClNpbmNlIEkgbmVlZCB0byBrbm93IGFsbCB0aGUgdXJscyBlYWNoIHVzZXIg aGFzIHZpc2l0ZWQsIEkgYW0gIHRlbXB0ZWQgdG8gY2FsbCB0aGUgZ3JvdXBCeUtleSBvbiB0aGlz IFJERC4gSG93ZXZlciwgc2luY2UgdGhlcmUgY291bGQgYmUgbWlsbGlvbnMgb2YgdXNlcnMgYW5k IHVybHMsIHRoZSBzaHVmZmxpbmcgY2F1c2VkIGJ5IGdyb3VwQnlLZXkgcHJvdmVzIHRvIGJlIGEg bWFqb3IgYm90dGxlbmVjayB0byBnZXQgdGhlIGpvYiBkb25lLiBJcyB0aGVyZSBhbnkgd29ya2Fy b3VuZD8gSSB3YW50IHRvIGVuZCB1cCB3aXRoIGFuIFJERCBvZiBrZXktdmFsdWUgcGFpcnMsIHdo ZXJlIHRoZSBrZXkgaXMgYW4gdXNlcl9pZCwgdGhlIHZhbHVlIGlzIGEgbGlzdCBvZiBhbGwgdGhl IHVybHMgdmlzaXRlZCBieSB0aGUgdXNlci4NCg0KVGhhbmtzLA0KDQpKaWFuZ3VvDQo= --_000_E34E3B513E144935A2402617023423ACgranturingcom_ Content-Type: text/html; charset="utf-8" Content-ID: <8D2D44CFE886F8409EDF6BF566E0BEC7@namprd07.prod.outlook.com> Content-Transfer-Encoding: base64 PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjwvaGVhZD4NCjxib2R5IHN0eWxlPSJ3b3JkLXdy YXA6IGJyZWFrLXdvcmQ7IC13ZWJraXQtbmJzcC1tb2RlOiBzcGFjZTsgLXdlYmtpdC1saW5lLWJy ZWFrOiBhZnRlci13aGl0ZS1zcGFjZTsgY29sb3I6IHJnYigwLCAwLCAwKTsgZm9udC1zaXplOiAx NHB4OyBmb250LWZhbWlseTogQ2FsaWJyaSwgc2Fucy1zZXJpZjsiPg0KPGRpdj4NCjxkaXY+WW91 IGNhbiB1c2UgYWdncmVnYXRlQnlLZXkgYXMgb25lIG9wdGlvbjo8L2Rpdj4NCjxkaXY+PGJyPg0K PC9kaXY+DQo8ZGl2PnZhbCBpbnB1dDogUkREW0ludCwgU3RyaW5nXSA9IC4uLjwvZGl2Pg0KPGRp dj48YnI+DQo8L2Rpdj4NCjxkaXY+dmFsIHRlc3QgPSBpbnB1dC5hZ2dyZWdhdGVCeUtleShMaXN0 QnVmZmVyLmVtcHR5W1N0cmluZ10pKChhLCBiKSA9Jmd0OyBhICYjNDM7PSBiLCAoYSwgYikgPSZn dDsgYSAmIzQzOyYjNDM7IGIpPC9kaXY+DQo8ZGl2Pg0KPGRpdiBpZD0iTUFDX09VVExPT0tfU0lH TkFUVVJFIj48L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pjxicj4NCjwvZGl2Pg0KPHNwYW4g aWQ9Ik9MS19TUkNfQk9EWV9TRUNUSU9OIj4NCjxkaXYgc3R5bGU9ImZvbnQtZmFtaWx5OkNhbGli cmk7IGZvbnQtc2l6ZToxMnB0OyB0ZXh0LWFsaWduOmxlZnQ7IGNvbG9yOmJsYWNrOyBCT1JERVIt Qk9UVE9NOiBtZWRpdW0gbm9uZTsgQk9SREVSLUxFRlQ6IG1lZGl1bSBub25lOyBQQURESU5HLUJP VFRPTTogMGluOyBQQURESU5HLUxFRlQ6IDBpbjsgUEFERElORy1SSUdIVDogMGluOyBCT1JERVIt VE9QOiAjYjVjNGRmIDFwdCBzb2xpZDsgQk9SREVSLVJJR0hUOiBtZWRpdW0gbm9uZTsgUEFERElO Ry1UT1A6IDNwdCI+DQo8c3BhbiBzdHlsZT0iZm9udC13ZWlnaHQ6Ym9sZCI+RnJvbTogPC9zcGFu PkppYW5ndW8gTGk8YnI+DQo8c3BhbiBzdHlsZT0iZm9udC13ZWlnaHQ6Ym9sZCI+RGF0ZTogPC9z cGFuPk1vbmRheSwgSnVuZSAyMiwgMjAxNSBhdCA1OjEyIFBNPGJyPg0KPHNwYW4gc3R5bGU9ImZv bnQtd2VpZ2h0OmJvbGQiPlRvOiA8L3NwYW4+JnF1b3Q7PGEgaHJlZj0ibWFpbHRvOnVzZXJAc3Bh cmsuYXBhY2hlLm9yZyI+dXNlckBzcGFyay5hcGFjaGUub3JnPC9hPiZxdW90Ozxicj4NCjxzcGFu IHN0eWxlPSJmb250LXdlaWdodDpib2xkIj5TdWJqZWN0OiA8L3NwYW4+d29ya2Fyb3VuZCBmb3Ig Z3JvdXBCeUtleTxicj4NCjwvZGl2Pg0KPGRpdj48YnI+DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0K PGRpdiBkaXI9Imx0ciI+SGksDQo8ZGl2Pjxicj4NCjwvZGl2Pg0KPGRpdj5JIGFtIHByb2Nlc3Np bmcgYW4gUkREIG9mIGtleS12YWx1ZSBwYWlycy4gVGhlIGtleSBpcyBhbiB1c2VyX2lkLCBhbmQg dGhlIHZhbHVlIGlzIGFuIHdlYnNpdGUgdXJsIHRoZSB1c2VyIGhhcyBldmVyIHZpc2l0ZWQuJm5i c3A7PC9kaXY+DQo8ZGl2Pjxicj4NCjwvZGl2Pg0KPGRpdj5TaW5jZSBJIG5lZWQgdG8ga25vdyBh bGwgdGhlIHVybHMgZWFjaCB1c2VyIGhhcyB2aXNpdGVkLCBJIGFtICZuYnNwO3RlbXB0ZWQgdG8g Y2FsbCB0aGUgZ3JvdXBCeUtleSBvbiB0aGlzIFJERC4gSG93ZXZlciwgc2luY2UgdGhlcmUgY291 bGQgYmUgbWlsbGlvbnMgb2YgdXNlcnMgYW5kIHVybHMsIHRoZSBzaHVmZmxpbmcgY2F1c2VkIGJ5 IGdyb3VwQnlLZXkgcHJvdmVzIHRvIGJlIGEgbWFqb3IgYm90dGxlbmVjayB0byBnZXQgdGhlIGpv YiBkb25lLg0KIElzIHRoZXJlIGFueSB3b3JrYXJvdW5kPyBJIHdhbnQgdG8gZW5kIHVwIHdpdGgg YW4gUkREIG9mIGtleS12YWx1ZSBwYWlycywgd2hlcmUgdGhlIGtleSBpcyBhbiB1c2VyX2lkLCB0 aGUgdmFsdWUgaXMgYSBsaXN0IG9mIGFsbCB0aGUgdXJscyB2aXNpdGVkIGJ5IHRoZSB1c2VyLjwv ZGl2Pg0KPGRpdj48YnI+DQo8L2Rpdj4NCjxkaXY+VGhhbmtzLDwvZGl2Pg0KPGRpdj48YnI+DQo8 L2Rpdj4NCjxkaXY+Smlhbmd1bzwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9zcGFu Pg0KPC9ib2R5Pg0KPC9odG1sPg0K --_000_E34E3B513E144935A2402617023423ACgranturingcom_--