Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 34721 invoked from network); 4 Aug 2010 18:21:47 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 4 Aug 2010 18:21:47 -0000 Received: (qmail 41032 invoked by uid 500); 4 Aug 2010 18:21:45 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 40652 invoked by uid 500); 4 Aug 2010 18:21:44 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 40525 invoked by uid 99); 4 Aug 2010 18:21:44 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Aug 2010 18:21:44 +0000 X-ASF-Spam-Status: No, hits=2.9 required=10.0 tests=HTML_MESSAGE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.215.172] (HELO mail-ey0-f172.google.com) (209.85.215.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Aug 2010 18:21:39 +0000 Received: by eyb7 with SMTP id 7so2355674eyb.31 for ; Wed, 04 Aug 2010 11:21:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.44.141 with SMTP id n13mr2483537web.16.1280946076719; Wed, 04 Aug 2010 11:21:16 -0700 (PDT) Received: by 10.216.19.137 with HTTP; Wed, 4 Aug 2010 11:21:16 -0700 (PDT) In-Reply-To: References: Date: Wed, 4 Aug 2010 11:21:16 -0700 Message-ID: Subject: Re: what causes MESSAGE-DESERIALIZER-POOL to spike From: Mike Malone To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0016365ee8a8e11bd4048d037ffe --0016365ee8a8e11bd4048d037ffe Content-Type: text/plain; charset=ISO-8859-1 This may be your problem: https://issues.apache.org/jira/browse/CASSANDRA-1358 The message deserializer executor is being created with a core pool size of 1. Since it uses a queue with unbounded capacity new requests are always queued and the thread pool never grows. So the message deserializer becomes a single-threaded bottleneck through which all traffic must pass. So your 16 cores are reduced to one core for handling all inter-node communication (and any intra-node communication that's being passed through the messaging service). Mike On Tue, Aug 3, 2010 at 10:02 PM, Dathan Pattishall wrote: > The output of htop shows threads as procs with a breakdown of how much cpu > /etc per thread (in ncurses color!). All of these Java "procs" are just Java > threads of only 1 instance of Cassandra per Server. > > > On Sat, Jul 31, 2010 at 3:45 PM, Benjamin Black wrote: > >> Sorry, I just noticed: are you running 14 instances of Cassandra on a >> single physical machine or are all those java processes something >> else? >> >> On Mon, Jul 26, 2010 at 12:22 PM, Dathan Pattishall >> wrote: >> > I have 4 nodes on enterprise type hardware (Lots of Ram 12GB, 16 i7 >> cores, >> > RAID Disks). >> > >> > ~# /opt/cassandra/bin/nodetool --host=localhost --port=8181 tpstats >> > Pool Name Active Pending Completed >> > STREAM-STAGE 0 0 0 >> > RESPONSE-STAGE 0 0 516280 >> > ROW-READ-STAGE 8 4096 1164326 >> > LB-OPERATIONS 0 0 0 >> > MESSAGE-DESERIALIZER-POOL 1 682008 1818682 >> > GMFD 0 0 6467 >> > LB-TARGET 0 0 0 >> > CONSISTENCY-MANAGER 0 0 661477 >> > ROW-MUTATION-STAGE 0 0 998780 >> > MESSAGE-STREAMING-POOL 0 0 0 >> > LOAD-BALANCER-STAGE 0 0 0 >> > FLUSH-SORTER-POOL 0 0 0 >> > MEMTABLE-POST-FLUSHER 0 0 4 >> > FLUSH-WRITER-POOL 0 0 4 >> > AE-SERVICE-STAGE 0 0 0 >> > HINTED-HANDOFF-POOL 0 0 3 >> > >> > EQX root@cass04:~# vmstat -n 1 >> > >> > procs -----------memory---------- ---swap-- -----io---- --system-- >> > -----cpu------ >> > r b swpd free buff cache si so bi bo in cs us sy >> id >> > wa st >> > 6 10 7096 121816 16244 10375492 0 0 1 3 0 0 5 >> 1 >> > 94 0 0 >> > 2 10 7096 116484 16248 10381144 0 0 5636 4 21210 9820 2 >> 1 >> > 79 18 0 >> > 1 9 7096 108920 16248 10387592 0 0 6216 0 21439 9878 2 >> 1 >> > 81 16 0 >> > 0 9 7096 129108 16248 10364852 0 0 6024 0 23280 8753 2 >> 1 >> > 80 17 0 >> > 2 9 7096 122460 16248 10370908 0 0 6072 0 20835 9461 2 >> 1 >> > 83 14 0 >> > 2 8 7096 115740 16260 10375752 0 0 5168 292 21049 9511 3 >> 1 >> > 77 20 0 >> > 1 10 7096 108424 16260 10382300 0 0 6244 0 21483 8981 2 >> 1 >> > 75 22 0 >> > 3 8 7096 125028 16260 10364104 0 0 5584 0 21238 9436 2 >> 1 >> > 81 16 0 >> > 3 9 7096 117928 16260 10370064 0 0 5988 0 21505 10225 >> 2 1 >> > 77 19 0 >> > 1 8 7096 109544 16260 10376640 0 0 6340 28 20840 8602 2 >> 1 >> > 80 18 0 >> > 0 9 7096 127028 16240 10357652 0 0 5984 0 20853 9158 2 >> 1 >> > 79 18 0 >> > 9 0 7096 121472 16240 10363492 0 0 5716 0 20520 8489 1 >> 1 >> > 82 16 0 >> > 3 9 7096 112668 16240 10369872 0 0 6404 0 21314 9459 2 >> 1 >> > 84 13 0 >> > 1 9 7096 127300 16236 10353440 0 0 5684 0 38914 10068 >> 2 1 >> > 76 21 0 >> > >> > >> > But the 16 cores are hardly utilized. Which indicates to me there is >> some >> > bad thread thrashing, but why? >> > >> > >> > >> > 1 [||||| 8.3%] >> Tasks: >> > 1070 total, 1 running >> > 2 [ 0.0%] Load >> > average: 8.34 9.05 8.82 >> > 3 [ 0.0%] >> Uptime: >> > 192 days(!), 15:29:52 >> > 4 [||||||||||| 17.9%] >> > 5 [||||| 5.7%] >> > 6 [|| 1.3%] >> > 7 [|| 2.6%] >> > 8 [| 0.6%] >> > 9 [| 0.6%] >> > 10 [|| 1.9%] >> > 11 [|| 1.9%] >> > 12 [|| 1.9%] >> > 13 [|| 1.3%] >> > 14 [| 0.6%] >> > 15 [|| 1.3%] >> > 16 [| 0.6%] >> > Mem[||||||||||||||||||||||||||||||||||||||||||||1791/12028MB] >> > Swp[| 6/1983MB] >> > >> > PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command >> > 30269 root 40 0 14100 2116 900 R 4.0 0.0 0:00.49 htop >> > 24878 root 40 0 20.6G 8345M 6883M D 3.0 69.4 1:23.03 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24879 root 40 0 20.6G 8345M 6883M D 3.0 69.4 1:22.93 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24874 root 40 0 20.6G 8345M 6883M D 2.0 69.4 1:22.73 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24880 root 40 0 20.6G 8345M 6883M D 2.0 69.4 1:22.93 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24875 root 40 0 20.6G 8345M 6883M D 2.0 69.4 1:23.17 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24658 root 40 0 20.6G 8345M 6883M D 2.0 69.4 1:23.06 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24877 root 40 0 20.6G 8345M 6883M S 2.0 69.4 1:23.43 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24873 root 40 0 20.6G 8345M 6883M D 1.0 69.4 1:23.65 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24876 root 40 0 20.6G 8345M 6883M S 1.0 69.4 1:23.62 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24942 root 40 0 20.6G 8345M 6883M S 1.0 69.4 0:23.50 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24943 root 40 0 20.6G 8345M 6883M S 0.0 69.4 0:29.53 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24933 root 40 0 20.6G 8345M 6883M S 0.0 69.4 0:22.57 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 24939 root 40 0 20.6G 8345M 6883M S 0.0 69.4 0:12.73 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > 25280 root 40 0 20.6G 8345M 6883M S 0.0 69.4 0:00.10 >> > /opt/java/bin/java -ea -Xms1G -Xmx7G -XX:+UseParNewGC -XX:+UseConcMark >> > >> > > --0016365ee8a8e11bd4048d037ffe Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: base64 VGhpcyBtYXkgYmUgeW91ciBwcm9ibGVtOqA8YSBocmVmPSJodHRwczovL2lzc3Vlcy5hcGFjaGUu b3JnL2ppcmEvYnJvd3NlL0NBU1NBTkRSQS0xMzU4Ij5odHRwczovL2lzc3Vlcy5hcGFjaGUub3Jn L2ppcmEvYnJvd3NlL0NBU1NBTkRSQS0xMzU4PC9hPjxkaXY+PGJyPjwvZGl2PjxkaXY+VGhlIG1l c3NhZ2UgZGVzZXJpYWxpemVyIGV4ZWN1dG9yIGlzIGJlaW5nIGNyZWF0ZWQgd2l0aCBhIGNvcmUg cG9vbCBzaXplIG9mIDEuIFNpbmNlIGl0IHVzZXMgYSBxdWV1ZSB3aXRoIHVuYm91bmRlZCBjYXBh Y2l0eSBuZXcgcmVxdWVzdHMgYXJlIGFsd2F5cyBxdWV1ZWQgYW5kIHRoZSB0aHJlYWQgcG9vbCBu ZXZlciBncm93cy4gU28gdGhlIG1lc3NhZ2UgZGVzZXJpYWxpemVyIGJlY29tZXMgYSBzaW5nbGUt dGhyZWFkZWQgYm90dGxlbmVjayB0aHJvdWdoIHdoaWNoIGFsbCB0cmFmZmljIG11c3QgcGFzcy4g U28geW91ciAxNiBjb3JlcyBhcmUgcmVkdWNlZCB0byBvbmUgY29yZSBmb3IgaGFuZGxpbmcgYWxs IGludGVyLW5vZGUgY29tbXVuaWNhdGlvbiAoYW5kIGFueSBpbnRyYS1ub2RlIGNvbW11bmljYXRp b24gdGhhdCYjMzk7cyBiZWluZyBwYXNzZWQgdGhyb3VnaCB0aGUgbWVzc2FnaW5nIHNlcnZpY2Up LjwvZGl2Pgo8ZGl2Pjxicj48L2Rpdj48ZGl2Pk1pa2U8YnI+PGJyPjxkaXYgY2xhc3M9ImdtYWls X3F1b3RlIj5PbiBUdWUsIEF1ZyAzLCAyMDEwIGF0IDEwOjAyIFBNLCBEYXRoYW4gUGF0dGlzaGFs bCA8c3BhbiBkaXI9Imx0ciI+Jmx0OzxhIGhyZWY9Im1haWx0bzpkYXRoYW52cEBnbWFpbC5jb20i PmRhdGhhbnZwQGdtYWlsLmNvbTwvYT4mZ3Q7PC9zcGFuPiB3cm90ZTo8YnI+PGJsb2NrcXVvdGUg Y2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFyZ2luOjAgMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6 MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleDsiPgpUaGUgb3V0cHV0IG9mIGh0b3Agc2hv d3MgdGhyZWFkcyBhcyBwcm9jcyB3aXRoIGEgYnJlYWtkb3duIG9mIGhvdyBtdWNoIGNwdSAvZXRj IHBlciB0aHJlYWQgKGluIG5jdXJzZXMgY29sb3IhKS4gQWxsIG9mIHRoZXNlIEphdmEgJnF1b3Q7 cHJvY3MmcXVvdDsgYXJlIGp1c3QgSmF2YSB0aHJlYWRzIG9mIG9ubHkgMSBpbnN0YW5jZSBvZiBD YXNzYW5kcmEgcGVyIFNlcnZlci48ZGl2PjxkaXY+CjwvZGl2PjxkaXYgY2xhc3M9Img1Ij48YnI+ PGJyPjxkaXYgY2xhc3M9ImdtYWlsX3F1b3RlIj4KT24gU2F0LCBKdWwgMzEsIDIwMTAgYXQgMzo0 NSBQTSwgQmVuamFtaW4gQmxhY2sgPHNwYW4gZGlyPSJsdHIiPiZsdDs8YSBocmVmPSJtYWlsdG86 YkBiM2sudXMiIHRhcmdldD0iX2JsYW5rIj5iQGIzay51czwvYT4mZ3Q7PC9zcGFuPiB3cm90ZTo8 YnI+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0iYm9yZGVyLWxlZnQ6MXB4 IHNvbGlkIHJnYigyMDQsIDIwNCwgMjA0KTttYXJnaW46MHB0IDBwdCAwcHQgMC44ZXg7cGFkZGlu Zy1sZWZ0OjFleCI+CgpTb3JyeSwgSSBqdXN0IG5vdGljZWQ6IGFyZSB5b3UgcnVubmluZyAxNCBp bnN0YW5jZXMgb2YgQ2Fzc2FuZHJhIG9uIGE8YnI+CnNpbmdsZSBwaHlzaWNhbCBtYWNoaW5lIG9y IGFyZSBhbGwgdGhvc2UgamF2YSBwcm9jZXNzZXMgc29tZXRoaW5nPGJyPgplbHNlPzxicj4KPGRp dj48ZGl2PjwvZGl2PjxkaXY+PGJyPgpPbiBNb24sIEp1bCAyNiwgMjAxMCBhdCAxMjoyMiBQTSwg RGF0aGFuIFBhdHRpc2hhbGwgJmx0OzxhIGhyZWY9Im1haWx0bzpkYXRoYW52cEBnbWFpbC5jb20i IHRhcmdldD0iX2JsYW5rIj5kYXRoYW52cEBnbWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8YnI+CiZn dDsgSSBoYXZlIDQgbm9kZXMgb24gZW50ZXJwcmlzZSB0eXBlIGhhcmR3YXJlIChMb3RzIG9mIFJh bSAxMkdCLCAxNiBpNyBjb3Jlcyw8YnI+CiZndDsgUkFJRCBEaXNrcykuPGJyPgomZ3Q7PGJyPgom Z3Q7IH4jIC9vcHQvY2Fzc2FuZHJhL2Jpbi9ub2RldG9vbCAtLWhvc3Q9bG9jYWxob3N0IC0tcG9y dD04MTgxIHRwc3RhdHM8YnI+CiZndDsgUG9vbCBOYW1loKCgoKCgoKCgoKCgoKCgoKCgoCBBY3Rp dmWgoCBQZW5kaW5noKCgoKAgQ29tcGxldGVkPGJyPgomZ3Q7IFNUUkVBTS1TVEFHRaCgoKCgoKCg oKCgoKCgoKCgoKCgoCAwoKCgoKCgoKAgMKCgoKCgoKCgoKCgoKAgMDxicj4KJmd0OyBSRVNQT05T RS1TVEFHRaCgoKCgoKCgoKCgoKCgoKCgoKAgMKCgoKCgoKCgIDCgoKCgoKCgoCA1MTYyODA8YnI+ CiZndDsgUk9XLVJFQUQtU1RBR0WgoKCgoKCgoKCgoKCgoKCgoKCgIDigoKCgoCA0MDk2oKCgoKCg oCAxMTY0MzI2PGJyPgomZ3Q7IExCLU9QRVJBVElPTlOgoKCgoKCgoKCgoKCgoKCgoKCgoCAwoKCg oKCgoKAgMKCgoKCgoKCgoKCgoKAgMDxicj4KJmd0OyBNRVNTQUdFLURFU0VSSUFMSVpFUi1QT09M oKCgoKCgoKAgMaCgoCA2ODIwMDigoKCgoKCgIDE4MTg2ODI8YnI+CiZndDsgR01GRKCgoKCgoKCg oKCgoKCgoKCgoKCgoKCgoKCgoKCgIDCgoKCgoKCgoCAwoKCgoKCgoKCgoCA2NDY3PGJyPgomZ3Q7 IExCLVRBUkdFVKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoCAwoKCgoKCgoKAgMKCgoKCgoKCgoKCg oKAgMDxicj4KJmd0OyBDT05TSVNURU5DWS1NQU5BR0VSoKCgoKCgoKCgoKCgoKAgMKCgoKCgoKCg IDCgoKCgoKCgoCA2NjE0Nzc8YnI+CiZndDsgUk9XLU1VVEFUSU9OLVNUQUdFoKCgoKCgoKCgoKCg oKCgIDCgoKCgoKCgoCAwoKCgoKCgoKAgOTk4NzgwPGJyPgomZ3Q7IE1FU1NBR0UtU1RSRUFNSU5H LVBPT0ygoKCgoKCgoKCgoCAwoKCgoKCgoKAgMKCgoKCgoKCgoKCgoKAgMDxicj4KJmd0OyBMT0FE LUJBTEFOQ0VSLVNUQUdFoKCgoKCgoKCgoKCgoKAgMKCgoKCgoKCgIDCgoKCgoKCgoKCgoKCgIDA8 YnI+CiZndDsgRkxVU0gtU09SVEVSLVBPT0ygoKCgoKCgoKCgoKCgoKCgIDCgoKCgoKCgoCAwoKCg oKCgoKCgoKCgoCAwPGJyPgomZ3Q7IE1FTVRBQkxFLVBPU1QtRkxVU0hFUqCgoKCgoKCgoKCgoCAw oKCgoKCgoKAgMKCgoKCgoKCgoKCgoKAgNDxicj4KJmd0OyBGTFVTSC1XUklURVItUE9PTKCgoKCg oKCgoKCgoKCgoKAgMKCgoKCgoKCgIDCgoKCgoKCgoKCgoKCgIDQ8YnI+CiZndDsgQUUtU0VSVklD RS1TVEFHRaCgoKCgoKCgoKCgoKCgoKCgIDCgoKCgoKCgoCAwoKCgoKCgoKCgoKCgoCAwPGJyPgom Z3Q7IEhJTlRFRC1IQU5ET0ZGLVBPT0ygoKCgoKCgoKCgoKCgoCAwoKCgoKCgoKAgMKCgoKCgoKCg oKCgoKAgMzxicj4KJmd0Ozxicj4KJmd0OyBFUVggcm9vdEBjYXNzMDQ6fiMgdm1zdGF0IC1uIDE8 YnI+CiZndDs8YnI+CiZndDsgcHJvY3MgLS0tLS0tLS0tLS1tZW1vcnktLS0tLS0tLS0tIC0tLXN3 YXAtLSAtLS0tLWlvLS0tLSAtLXN5c3RlbS0tPGJyPgomZ3Q7IC0tLS0tY3B1LS0tLS0tPGJyPgom Z3Q7IKByoCBioKAgc3dwZKCgIGZyZWWgoCBidWZmoCBjYWNoZaCgIHNpoKAgc2+goKAgYmmgoKAg Ym+goCBpbqCgIGNzIHVzIHN5IGlkPGJyPgomZ3Q7IHdhIHN0PGJyPgomZ3Q7IKA2IDEwoKAgNzA5 NiAxMjE4MTagIDE2MjQ0IDEwMzc1NDkyoKCgIDCgoKAgMKCgoKAgMaCgoKAgM6CgoCAwoKCgIDCg IDWgIDE8YnI+CiZndDsgOTSgIDCgIDA8YnI+CiZndDsgoDIgMTCgoCA3MDk2IDExNjQ4NKAgMTYy NDggMTAzODExNDSgoKAgMKCgoCAwoCA1NjM2oKCgoCA0IDIxMjEwIDk4MjCgIDKgIDE8YnI+CiZn dDsgNzkgMTigIDA8YnI+CiZndDsgoDGgIDmgoCA3MDk2IDEwODkyMKAgMTYyNDggMTAzODc1OTKg oKAgMKCgoCAwoCA2MjE2oKCgoCAwIDIxNDM5IDk4NzigIDKgIDE8YnI+CiZndDsgODEgMTagIDA8 YnI+CiZndDsgoDCgIDmgoCA3MDk2IDEyOTEwOKAgMTYyNDggMTAzNjQ4NTKgoKAgMKCgoCAwoCA2 MDI0oKCgoCAwIDIzMjgwIDg3NTOgIDKgIDE8YnI+CiZndDsgODAgMTegIDA8YnI+CiZndDsgoDKg IDmgoCA3MDk2IDEyMjQ2MKAgMTYyNDggMTAzNzA5MDigoKAgMKCgoCAwoCA2MDcyoKCgoCAwIDIw ODM1IDk0NjGgIDKgIDE8YnI+CiZndDsgODMgMTSgIDA8YnI+CiZndDsgoDKgIDigoCA3MDk2IDEx NTc0MKAgMTYyNjAgMTAzNzU3NTKgoKAgMKCgoCAwoCA1MTY4oKAgMjkyIDIxMDQ5IDk1MTGgIDOg IDE8YnI+CiZndDsgNzcgMjCgIDA8YnI+CiZndDsgoDEgMTCgoCA3MDk2IDEwODQyNKAgMTYyNjAg MTAzODIzMDCgoKAgMKCgoCAwoCA2MjQ0oKCgoCAwIDIxNDgzIDg5ODGgIDKgIDE8YnI+CiZndDsg NzUgMjKgIDA8YnI+CiZndDsgoDOgIDigoCA3MDk2IDEyNTAyOKAgMTYyNjAgMTAzNjQxMDSgoKAg MKCgoCAwoCA1NTg0oKCgoCAwIDIxMjM4IDk0MzagIDKgIDE8YnI+CiZndDsgODEgMTagIDA8YnI+ CiZndDsgoDOgIDmgoCA3MDk2IDExNzkyOKAgMTYyNjAgMTAzNzAwNjSgoKAgMKCgoCAwoCA1OTg4 oKCgoCAwIDIxNTA1IDEwMjI1oCAyoCAxPGJyPgomZ3Q7IDc3IDE5oCAwPGJyPgomZ3Q7IKAxoCA4 oKAgNzA5NiAxMDk1NDSgIDE2MjYwIDEwMzc2NjQwoKCgIDCgoKAgMKAgNjM0MKCgoCAyOCAyMDg0 MCA4NjAyoCAyoCAxPGJyPgomZ3Q7IDgwIDE4oCAwPGJyPgomZ3Q7IKAwoCA5oKAgNzA5NiAxMjcw MjigIDE2MjQwIDEwMzU3NjUyoKCgIDCgoKAgMKAgNTk4NKCgoKAgMCAyMDg1MyA5MTU4oCAyoCAx PGJyPgomZ3Q7IDc5IDE4oCAwPGJyPgomZ3Q7IKA5oCAwoKAgNzA5NiAxMjE0NzKgIDE2MjQwIDEw MzYzNDkyoKCgIDCgoKAgMKAgNTcxNqCgoKAgMCAyMDUyMCA4NDg5oCAxoCAxPGJyPgomZ3Q7IDgy IDE2oCAwPGJyPgomZ3Q7IKAzoCA5oKAgNzA5NiAxMTI2NjigIDE2MjQwIDEwMzY5ODcyoKCgIDCg oKAgMKAgNjQwNKCgoKAgMCAyMTMxNCA5NDU5oCAyoCAxPGJyPgomZ3Q7IDg0IDEzoCAwPGJyPgom Z3Q7IKAxoCA5oKAgNzA5NiAxMjczMDCgIDE2MjM2IDEwMzUzNDQwoKCgIDCgoKAgMKAgNTY4NKCg oKAgMCAzODkxNCAxMDA2OKAgMqAgMTxicj4KJmd0OyA3NiAyMaAgMDxicj4KJmd0Ozxicj4KJmd0 Ozxicj4KJmd0OyBCdXQgdGhlIDE2IGNvcmVzIGFyZSBoYXJkbHkgdXRpbGl6ZWQuIFdoaWNoIGlu ZGljYXRlcyB0byBtZSB0aGVyZSBpcyBzb21lPGJyPgomZ3Q7IGJhZCB0aHJlYWQgdGhyYXNoaW5n LCBidXQgd2h5Pzxicj4KJmd0Ozxicj4KJmd0Ozxicj4KJmd0Ozxicj4KJmd0OyCgIDGgIFt8fHx8 fKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKAgOC4zJV2goKCg IFRhc2tzOjxicj4KJmd0OyAxMDcwIHRvdGFsLCAxIHJ1bm5pbmc8YnI+CiZndDsgoCAyoCBboKCg oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgIDAuMCVdoKCg oCBMb2FkPGJyPgomZ3Q7IGF2ZXJhZ2U6IDguMzQgOS4wNSA4LjgyPGJyPgomZ3Q7IKAgM6AgW6Cg oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoCAwLjAlXaCg oKAgVXB0aW1lOjxicj4KJmd0OyAxOTIgZGF5cyghKSwgMTU6Mjk6NTI8YnI+CiZndDsgoCA0oCBb fHx8fHx8fHx8fHygoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKAgMTcuOSVd PGJyPgomZ3Q7IKAgNaAgW3x8fHx8oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCg oKCgoKCgoKCgoCA1LjclXTxicj4KJmd0OyCgIDagIFt8fKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCg oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKAgMS4zJV08YnI+CiZndDsgoCA3oCBbfHygoKCgoKCg oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgIDIuNiVdPGJyPgomZ3Q7 IKAgOKAgW3ygoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCg oCAwLjYlXTxicj4KJmd0OyCgIDmgIFt8oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCg oKCgoKCgoKCgoKCgoKCgoKAgMC42JV08YnI+CiZndDsgoCAxMCBbfHygoKCgoKCgoKCgoKCgoKCg oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgIDEuOSVdPGJyPgomZ3Q7IKAgMTEgW3x8 oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoCAxLjklXTxi cj4KJmd0OyCgIDEyIFt8fKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCg oKCgoKCgoKAgMS45JV08YnI+CiZndDsgoCAxMyBbfHygoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCg oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgIDEuMyVdPGJyPgomZ3Q7IKAgMTQgW3ygoKCgoKCgoKCg oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoCAwLjYlXTxicj4KJmd0OyCg IDE1IFt8fKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKAg MS4zJV08YnI+CiZndDsgoCAxNiBbfKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCg oKCgoKCgoKCgoKCgoKCgIDAuNiVdPGJyPgomZ3Q7IKAgTWVtW3x8fHx8fHx8fHx8fHx8fHx8fHx8 fHx8fHx8fHx8fHx8fHx8fHx8fHx8fHx8MTc5MS8xMjAyOE1CXTxicj4KJmd0OyCgIFN3cFt8oKCg oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoCA2LzE5ODNNQl08YnI+ CiZndDs8YnI+CiZndDsgoCBQSUQgVVNFUqCgoKAgUFJJoCBOSaAgVklSVKCgIFJFU6CgIFNIUiBT IENQVSUgTUVNJaCgIFRJTUUroCBDb21tYW5kPGJyPgomZ3Q7IDMwMjY5IHJvb3SgoKCgoCA0MKCg IDAgMTQxMDCgIDIxMTagoCA5MDAgUqAgNC4woCAwLjCgIDA6MDAuNDkgaHRvcDxicj4KJmd0OyAy NDg3OCByb290oKCgoKAgNDCgoCAwIDIwLjZHIDgzNDVNIDY4ODNNIESgIDMuMCA2OS40oCAxOjIz LjAzPGJyPgomZ3Q7IC9vcHQvamF2YS9iaW4vamF2YSAtZWEgLVhtczFHIC1YbXg3RyAtWFg6K1Vz ZVBhck5ld0dDIC1YWDorVXNlQ29uY01hcms8YnI+CiZndDsgMjQ4Nzkgcm9vdKCgoKCgIDQwoKAg MCAyMC42RyA4MzQ1TSA2ODgzTSBEoCAzLjAgNjkuNKAgMToyMi45Mzxicj4KJmd0OyAvb3B0L2ph dmEvYmluL2phdmEgLWVhIC1YbXMxRyAtWG14N0cgLVhYOitVc2VQYXJOZXdHQyAtWFg6K1VzZUNv bmNNYXJrPGJyPgomZ3Q7IDI0ODc0IHJvb3SgoKCgoCA0MKCgIDAgMjAuNkcgODM0NU0gNjg4M00g RKAgMi4wIDY5LjSgIDE6MjIuNzM8YnI+CiZndDsgL29wdC9qYXZhL2Jpbi9qYXZhIC1lYSAtWG1z MUcgLVhteDdHIC1YWDorVXNlUGFyTmV3R0MgLVhYOitVc2VDb25jTWFyazxicj4KJmd0OyAyNDg4 MCByb290oKCgoKAgNDCgoCAwIDIwLjZHIDgzNDVNIDY4ODNNIESgIDIuMCA2OS40oCAxOjIyLjkz PGJyPgomZ3Q7IC9vcHQvamF2YS9iaW4vamF2YSAtZWEgLVhtczFHIC1YbXg3RyAtWFg6K1VzZVBh ck5ld0dDIC1YWDorVXNlQ29uY01hcms8YnI+CiZndDsgMjQ4NzUgcm9vdKCgoKCgIDQwoKAgMCAy MC42RyA4MzQ1TSA2ODgzTSBEoCAyLjAgNjkuNKAgMToyMy4xNzxicj4KJmd0OyAvb3B0L2phdmEv YmluL2phdmEgLWVhIC1YbXMxRyAtWG14N0cgLVhYOitVc2VQYXJOZXdHQyAtWFg6K1VzZUNvbmNN YXJrPGJyPgomZ3Q7IDI0NjU4IHJvb3SgoKCgoCA0MKCgIDAgMjAuNkcgODM0NU0gNjg4M00gRKAg Mi4wIDY5LjSgIDE6MjMuMDY8YnI+CiZndDsgL29wdC9qYXZhL2Jpbi9qYXZhIC1lYSAtWG1zMUcg LVhteDdHIC1YWDorVXNlUGFyTmV3R0MgLVhYOitVc2VDb25jTWFyazxicj4KJmd0OyAyNDg3NyBy b290oKCgoKAgNDCgoCAwIDIwLjZHIDgzNDVNIDY4ODNNIFOgIDIuMCA2OS40oCAxOjIzLjQzPGJy PgomZ3Q7IC9vcHQvamF2YS9iaW4vamF2YSAtZWEgLVhtczFHIC1YbXg3RyAtWFg6K1VzZVBhck5l d0dDIC1YWDorVXNlQ29uY01hcms8YnI+CiZndDsgMjQ4NzMgcm9vdKCgoKCgIDQwoKAgMCAyMC42 RyA4MzQ1TSA2ODgzTSBEoCAxLjAgNjkuNKAgMToyMy42NTxicj4KJmd0OyAvb3B0L2phdmEvYmlu L2phdmEgLWVhIC1YbXMxRyAtWG14N0cgLVhYOitVc2VQYXJOZXdHQyAtWFg6K1VzZUNvbmNNYXJr PGJyPgomZ3Q7IDI0ODc2IHJvb3SgoKCgoCA0MKCgIDAgMjAuNkcgODM0NU0gNjg4M00gU6AgMS4w IDY5LjSgIDE6MjMuNjI8YnI+CiZndDsgL29wdC9qYXZhL2Jpbi9qYXZhIC1lYSAtWG1zMUcgLVht eDdHIC1YWDorVXNlUGFyTmV3R0MgLVhYOitVc2VDb25jTWFyazxicj4KJmd0OyAyNDk0MiByb290 oKCgoKAgNDCgoCAwIDIwLjZHIDgzNDVNIDY4ODNNIFOgIDEuMCA2OS40oCAwOjIzLjUwPGJyPgom Z3Q7IC9vcHQvamF2YS9iaW4vamF2YSAtZWEgLVhtczFHIC1YbXg3RyAtWFg6K1VzZVBhck5ld0dD IC1YWDorVXNlQ29uY01hcms8YnI+CiZndDsgMjQ5NDMgcm9vdKCgoKCgIDQwoKAgMCAyMC42RyA4 MzQ1TSA2ODgzTSBToCAwLjAgNjkuNKAgMDoyOS41Mzxicj4KJmd0OyAvb3B0L2phdmEvYmluL2ph dmEgLWVhIC1YbXMxRyAtWG14N0cgLVhYOitVc2VQYXJOZXdHQyAtWFg6K1VzZUNvbmNNYXJrPGJy PgomZ3Q7IDI0OTMzIHJvb3SgoKCgoCA0MKCgIDAgMjAuNkcgODM0NU0gNjg4M00gU6AgMC4wIDY5 LjSgIDA6MjIuNTc8YnI+CiZndDsgL29wdC9qYXZhL2Jpbi9qYXZhIC1lYSAtWG1zMUcgLVhteDdH IC1YWDorVXNlUGFyTmV3R0MgLVhYOitVc2VDb25jTWFyazxicj4KJmd0OyAyNDkzOSByb290oKCg oKAgNDCgoCAwIDIwLjZHIDgzNDVNIDY4ODNNIFOgIDAuMCA2OS40oCAwOjEyLjczPGJyPgomZ3Q7 IC9vcHQvamF2YS9iaW4vamF2YSAtZWEgLVhtczFHIC1YbXg3RyAtWFg6K1VzZVBhck5ld0dDIC1Y WDorVXNlQ29uY01hcms8YnI+CiZndDsgMjUyODAgcm9vdKCgoKCgIDQwoKAgMCAyMC42RyA4MzQ1 TSA2ODgzTSBToCAwLjAgNjkuNKAgMDowMC4xMDxicj4KJmd0OyAvb3B0L2phdmEvYmluL2phdmEg LWVhIC1YbXMxRyAtWG14N0cgLVhYOitVc2VQYXJOZXdHQyAtWFg6K1VzZUNvbmNNYXJrPGJyPgom Z3Q7PGJyPgo8L2Rpdj48L2Rpdj48L2Jsb2NrcXVvdGU+PC9kaXY+PGJyPgo8L2Rpdj48L2Rpdj48 L2Jsb2NrcXVvdGU+PC9kaXY+PGJyPjwvZGl2Pgo= --0016365ee8a8e11bd4048d037ffe--