From dev-return-74883-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Sun Oct 21 18:43:04 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4E29818065D for ; Sun, 21 Oct 2018 18:43:03 +0200 (CEST) Received: (qmail 25660 invoked by uid 500); 21 Oct 2018 16:43:02 -0000 Mailing-List: contact dev-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zookeeper.apache.org Delivered-To: mailing list dev@zookeeper.apache.org Delivered-To: moderator for dev@zookeeper.apache.org Received: (qmail 50111 invoked by uid 99); 21 Oct 2018 06:12:22 -0000 X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.105 X-Spam-Level: ** X-Spam-Status: No, score=2.105 tagged_above=-999 required=6.31 tests=[FROM_EXCESS_BASE64=0.105, HTML_IMAGE_RATIO_08=0.001, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled X-AuditID: ca6c0e64-acdff700000013a7-b6-5bcc18aa20b1 From: =?gb2312?B?zO/S48i6?= To: "dev@zookeeper.apache.org" Subject: ZooKeeper two issues review Thread-Topic: ZooKeeper two issues review Thread-Index: AdRpBPAPhLyE4Z6XR+a2ZOwPcEsu7A== Date: Sun, 21 Oct 2018 06:11:53 +0000 Message-ID: <5d4de9b22fc04735857a524c977a1b21@EXCH10.iqiyi.pps> Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.121.90.26] Content-Type: multipart/related; boundary="_005_5d4de9b22fc04735857a524c977a1b21EXCH10iqiyipps_"; type="multipart/alternative" MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLKsWRmVeSWpSXmKPExsXCVamgprta4ky0wZY3/BZf5h5gd2D0OLSw kyWAMYrbJimxpCw4Mz1P3y6BO+Ndw0umghXHmSu2fZnO0sD4fxNzFyMHh4SAicTyLVldjFwc QgKbGCWmvuwHinNysAkYSOx4vB2sRkTAUuL1MgeQsLCAisSMiRdYQWwRAU2Jlg3bWCBsPYlv L18wgdgsAqoSXbsWgMV5Bewl9rycDjaSUUBWYtqj+2A1zALiEnOnzQKbIyEgIvHw4mk2CFtU 4uXjf1BxA4mtS/exQNgKEu+2HGcEuZNZoIlR4vT9e8wQCwQlTs58AlYkJCAvseliP8sERqFZ SHbMQtYzC0kPRFGaxPO/3YwQtpbEvIbfTBC2osSU7ofsELamxJXJh6BsbYllC18zQ9i6Ev// nmeDqX9z+Q0jxLJljBLPz3UAJTiAHB2Jx9s4Mc3UkZjw9B4rTHzDnn6oOFDv9iliMDX725ax IOtdwCi0ilGoODddNylL18BIrzCzMlMvOT93EyMwNZzK4UvZwXhwvvMhRgEORiUe3gWTTkcL sSaWFVfmHmJUARryaMPqC4xSLHn5ealKIryzTgCleVMSK6tSi/Lji0pzUosPMUpzsCiJ8+7o OBktJJCeWJKanZpakFoEk2Xi4DzEKMHBJSVSnJqXklqUWFqSEQ9KZ/HFwIQm1cDIbpI1ryLg 0pmZvowZ5ruN56UncMTX+7QyeqdVR0mI/1skvkGhfM7yu6511/b9E3l3afpBJVPdYzGF3760 3FyR+6+g8LSFXK/Q3M8e6zw+Hde8E5QR2T79lFlGgl6Jg6/mHE/D7CdLQ/P2fbS2n9JoNzF/ p8o526/Pw62DQlezNr9WFvJ271JiKc5INNRiLipOBADdvc3QMAMAAA== --_005_5d4de9b22fc04735857a524c977a1b21EXCH10iqiyipps_ Content-Type: multipart/alternative; boundary="_000_5d4de9b22fc04735857a524c977a1b21EXCH10iqiyipps_" --_000_5d4de9b22fc04735857a524c977a1b21EXCH10iqiyipps_ Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: base64 SGksIGFsbA0KSSBwcm9wb3NlZCBhIEppcmEgaXNzdWUgdG8gY29tbWl0IFpvb0tlZXBlciBjb2Rl cy4gSSB3YXMgYXNrZWQgdG8gZm9sbG93IHRoZSBuZXcgaXNzdWUuIFNvIGZpcnN0bHkgSSBuZWVk IHRvIHNlbmQgYW4gZW1haWwgdG8gZGVzY3JpYmUgbXkgdHdvIGlzc3Vlcy4NCg0KRmlyc3Qgb25l Og0KSmlyYTogIGh0dHBzOi8vaXNzdWVzLmFwYWNoZS5vcmcvamlyYS9icm93c2UvWk9PS0VFUEVS LTMxNjcuDQpQdXJwb3NlOiAgYWRkIGFuIEFQSSB0byBnZXQgdG90YWwgY291bnQgb2YgcmVjdXJz aXZlIHN1YiBub2RlcyBvZiBvbmUgbm9kZQ0KRGVzY3JpcHRpb246DQoNCjEuICAgIEluIHByb2R1 Y3Rpb24gZW52aXJvbm1lbnQsIHRoZXJlIHdpbGwgYmUgYWx3YXlzIGEgc2l0dWF0aW9uIHRoYXQg dGhlcmUgYXJlIGEgbG90IG9mIHJlY3Vyc2l2ZSBzdWIgbm9kZXMgb2Ygb25lIG5vZGUuIFdlIG5l ZWQgdG8gY291bnQgdG90YWwgbnVtYmVyIG9mIHRoZSBub2RlLiBMaWtlIHRoaXMuo6hXZSB3YW50 IHRvIGdldCBhbGwgdGhlIHN1Ym5vZGVzIG9mIG5vZGVBLqOpDQoNCltjaWQ6aW1hZ2UwMDIuanBn QDAxRDQ2OTQ4LjAzRjI3NDkwXQ0KDQoyLiBOb3csIHdlIGNhbiBvbmx5IHVzZSBBUEkgZ2V0Q2hp bGRyZW4gd2hpY2ggcmV0dXJucyB0aGUgTGlzdDxTdHJpbmc+IG9mIGZpcnN0IGxldmVsIG9mIHN1 YiBub2Rlcy4oV2UgY2FuIG9ubHkgZ2V0IHRoZSBub2RlQiBsaXN0IGRpcmVjdGx5KS4gV2UgbmVl ZCB0byBpdGVyYXRlIGV2ZXJ5IHN1YiBub2RlIHRvIGdldCByZWN1cnNpdmUgc3ViIG5vZGVzLiBJ dCB3aWxsIGNvc3QgYSBsb3Qgb2YgdGltZS4NCg0KMy4gSW4gem9va2VlcGVyIHNlcnZlciBzaWRl LCBpdCB1c2VzIEhhc3A8U3RyaW5nLCBEYXRhTm9kZT4gdG8gc3RvcmUgbm9kZS4gVGhlIGtleSBv ZiB0aGUgbWFwIHJlcHJlc2VudHMgdGhlIHBhdGggb2YgdGhlIG5vZGUuIFdlIGNhbiBpdGVyYXRl IHRoZSBtYXAgZ2V0IHRvdGFsIG51bWJlciBvZiBhbGwgbGV2ZWxzIG9mIHN1YiBub2RlcyBvZiBv bmUgbm9kZS4NCg0KDQpTZWNvbmQgT25lOg0KSmlyYTogIGh0dHBzOi8vaXNzdWVzLmFwYWNoZS5v cmcvamlyYS9icm93c2UvWk9PS0VFUEVSLTMxNjgNClB1cnBvc2U6ICBSZWR1Y2Ugc2Vzc2lvbiBy ZXZhbGlkYXRpb24gdGltZSBhZnRlciB6eGlkIHJvbGwgb3Zlcg0KRGVzY3JpcHRpb246DQoNCjEu IFNvbWV0aW1lcyBab29rZWVwZXIgY2x1c3RlciB3aWxsIHJlY2VpdmUgYSBsb3Qgb2YgY29ubmVj dGlvbnMgZnJvbSBjbGllbnRzLCBzb21ldGltZXMgY29ubmVjdGlvbiBudW1iZXIgZXZlbiBleGNl ZWRzIDFXLiBXaGVuIHp4aWQgcm9sbHMgb3ZlciwgdGhlIGNsaWVudHMgd2lsbCByZWNvbm5lY3Qg YW5kIHJldmFsaWRhdGUgdGhlIHNlc3Npb24uDQoNCjIuIEluIFpvb2tlZXBlciBkZXNpZ24gc3Ry dWN0dXJlLCB3aGVuIGZvbGxvd2VyIHNlcnZlciByZWNlaXZlcyB0aGUgc2Vzc2lvbiByZXZhbGlk YXRpb24gcmVxdWVzdHMsIGl0IHdpbGwgc2VuZCByZXF1ZXN0cyB0byBsZWFkZXIgc2VydmVyLCB3 aGljaCBpcyBkZXNpZ25lZCB0byBiZSByZXNwb25zaWJsZSBmb3Igc2Vzc2lvbiByZXZhbGlkYXRp b24uDQoNCltjaWQ6aW1hZ2UwMDQucG5nQDAxRDQ2OTQ4LjAzRjI3NDkwXSBXaGVuIExlYXJuZXJa b29LZWVwZXJTZXJ2ZXIgcmVjZWl2ZXMgcmVjb25uZWN0aW9uLCBpdCB3aWxsIHNlbmQgcmV2YWxp ZGF0aW9uIHJlcXVlc3RzIHRvIExlYWRlclpvb0tlZXBlclNlcnZlci4gTGVhZGVyWm9vS2VlcGVy U2VydmVyIHdpbGwgZmFjZSBhIGxvdCBvZiBwcmVzc3VyZS4NCg0KMy4gSW4gYSBzaG9ydCB0aW1l LCBMZWFkZXIgd2lsbCBoYW5kbGUgbG90cyBvZiByZXF1ZXN0cy4gSSB1c2UgYSB0b29sIHRvIGdl dCB0aGUgc3RhdGlzdGljcywgc29tZSBjbGllbnRzIG5lZWQgdG8gd2FpdCBvdmVyIDIwcy4gSXQg aXMgdG9vIGxvbmcgZm9yIHNvbWUgc3BlY2lhbCBjbGllbnRzLCBsaWtlIFJlc291cmNlTWFuYWdl ci4NCg0KNC4gSSBkZXNpZ24gYSB0aG91Z2h0OiB3aGVuIHp4aWQgcm9sbG92ZXIgaGFwcGVucy4g TGVhZGVyIHdpbGwgcmVjb3JkIHRoZSBhY2N1cmF0ZSB0aW1lLiBXaGVuIHJlZWxlY3Rpb24gZmlu aXNocywgYWxsIHNlcnZlcnMgd2lsbCBnZXQgdGhlIHJvbGxvdmVyIHRpbWUuIFdoZW4gY2xpZW50 cyByZWNvbm5lY3QgYW5kIHJldmFsaWRhdGUgc2Vzc2lvbi4gQWxsIHNlcnZlcnMgY2FuIGp1ZGdl IGl0LiBTbyBpdCBjYW4gcmVkdWNlIGEgbG90cyBvZiBwcmVzc3VyZSBvZiBjbHVzdGVyLCBhbGwg Y2xpZW50cyBjYW4gd2lsbCB3YWl0IGZvciBsZXNzIHRpbWUuDQoNCg0KVGhlc2UgYXJlIG15IHR3 byBpc3N1ZXMuIEhlbHAgdG8gcmV2aWV3IHRoZSBzb2x1dGlvbiBpcyByaWdodCBvciBub3QuIFRo YW5rIHlvdSBhIGxvdC4NCszv0uPIug0KvLzK9bL6xrfW0NDEICDUxsa9zKgNCrCuxubS1bmry74N ClFJWUkuY29tLCBJbmMuDQq12Na3o7rJz7qjytCzpMT+x/jB2brnwrczNjW6xbCuxubS1bS00MK0 88/DNrLjDQrTyrHgo7oyMDExMDMNCsrWu/qjuqOrODYgMTU3IDIxNDAgMTI1Ng0K08rP5KO6dGlh bnlpcXVuQHFpeWkuY29tPG1haWx0bzp0aWFueWlxdW5AcWl5aS5jb20+DQoNCg== --_000_5d4de9b22fc04735857a524c977a1b21EXCH10iqiyipps_ Content-Type: text/html; charset="gb2312" Content-Transfer-Encoding: quoted-printable

Hi, = all

I pr= oposed a Jira issue to commit ZooKeeper codes. I was asked to follow the ne= w issue. So firstly I need to send an email to describe my two issues.=

 

F= irst one:  

J= ira:  https://issues.= apache.org/jira/browse/ZOOKEEPER-3167.

P= urpose:  ad= d an API to get total count of recursive sub nodes of one node

D= escription:

1.   = ; In production environ= ment, there will be always a situation that there are a lot of recursive su= b nodes of one node. We need to count total number of the node. Like this.=A3=A8We want to get all the subnodes of nodeA= .=A3=A9

2. Now, we can only use API getChildren which return= s the List<String> of first level of sub nodes.(We can only get= the nodeB list directly). We need to iterate every sub node to get recursive sub nodes. It will cost a lot of time.

3. In zookeeper server side, it uses Hasp<String,= DataNode> to store node. The key of the map represents the path of the = node. We can iterate the map get total number of all levels of sub nodes of one node.

 

 

S= econd One:

J= ira:  https://is= sues.apache.org/jira/browse/ZOOKEEPER-3168

P= urpose:  Re= duce session revalidation time after zxid roll over

D= escription:

1. Sometimes Zo= okeeper cluster will receive a lot of connections from clients, sometimes connection numbe= r even exceeds 1W. When zxid rolls over, the clients will reconnect and rev= alidate the session.

2. In Zookeeper design structure, when follower serv= er receives the session revalidation requests, it will send requests to lea= der server, which is designed to be responsible for session revalidation.

When LearnerZooKeeperServer receives reconnection, it will send reval= idation requests to LeaderZooKeeperServer. LeaderZooKeeperServer will face = a lot of pressure.

3. In a short time, Leader will handle lots of reque= sts. I use a tool to get the statistics, some clients need to wait over 20s= . It is too long for some special clients, like ResourceManager.=

4. I design a thought: when zxid rollover happens. L= eader will record the accurate time. When reelection finishs, all servers w= ill get the rollover time. When clients reconnect and revalidate session. All servers can judge it. So it can reduce a lots of pressure of = cluster, all clients can will wait for less time.

<= o:p> 

 

These are= my two issues. Help to review the solution is right or not. Thank you a lo= t.

=CC=EF=D2=E3=C8=BA

=BC=BC=CA=F5=B2=FA=C6=B7= =D6=D0=D0=C4  =D4=C6=C6=BD=CC=A8

=B0=AE=C6=E6=D2=D5=B9=AB= =CB=BE 

QIYI.com, Inc.

=B5=D8=D6= =B7=A3=BA=C9=CF=BA=A3=CA=D0=B3=A4=C4=FE=C7=F8=C1=D9=BA=E7=C2=B7365=BA=C5=B0=AE=C6=E6=D2=D5=B4=B4=D0=C2=B4=F3=CF=C36=B2=E3=

=D3=CA=B1= =E0=A3=BA201103

=CA=D6=BB= =FA=A3=BA=A3=AB86 157 2140 1256

=D3=CA=CF= =E4=A3=BA= tianyiqun@qiyi.com

 

--_000_5d4de9b22fc04735857a524c977a1b21EXCH10iqiyipps_-- --_005_5d4de9b22fc04735857a524c977a1b21EXCH10iqiyipps_--