Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
MIME-Version: 1.0
In-Reply-To: <etPan.572cc0df.7aa2863b.1e0@Hackintosh.local>
References: <etPan.572cc0df.7aa2863b.1e0@Hackintosh.local>
Date: Fri, 20 May 2016 19:14:47 +0200
Message-ID: <CAJgLgcizYzmGjkP1=6gFNZLgHHKXc6Pg1J_E-GUariyKJzXMHw@mail.gmail.com>
Subject: Re: SSTable count at 10K during repair (won't decrease)
From: Fabrice Facorat <fabrice.facorat@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=001a113d033e4e81970533493aad
archived-at: Fri, 20 May 2016 17:14:54 -0000

--001a113d033e4e81970533493aad
Content-Type: text/plain; charset=UTF-8

Are you using repairParallelism = sequential or parallel ?

As said by Alain:
- try to decrease streamthroughput to avoid overflooding nodes with a lots
of (small) streamed sstables
- if you are using // repair, switch to sequential
- don't start too much repair simultaneously.
- Do you really need to use LCS for your tables ? LCS make the problem even
worse. Use it with parcimony ;)


2016-05-06 18:05 GMT+02:00 Jean-Francois Gosselin <jfgosselin@gmail.com>:

> - Cassandra 2.1.13
> - SSDs
> - LeveledCompactionStrategy
> - Range repair (not incremental) with Spotify's Reaper
> https://github.com/spotify/cassandra-reaper
>
> Problem : When we run a repair job sometimes the SSTable count goes to 10K
> on one of nodes (not always the same node). The Reaper is smart enough to
> postpone the repair on this node since the number of pending compactions is
> > 20 but number of SSTables stays around 10K.
> Even If I set the compactionthroughput 0 (disable throttling) the SSTable
> count stays around 10K.
>
> Workaround: If we abort the repair, and restart the node it quickly (in 15
> minutes) goes back to 200 SSTables ...
>
> Any suggestions as what I should look at ?
>
> When it occurs, I've noticed that nodetool compactionstats and cfstats (on
> the table with 10K SSTables) takes minutes to return with a result.
>
> I thought that the issue might be related to
> https://issues.apache.org/jira/browse/CASSANDRA-10766 as I see the
> MemtablePostFlush waiting on the countdown latch but the Pending
> MemtablePostFlush is going up and down according to tpstats.
>
> Complete stack trace : http://pastebin.com/K1r3CUff
>
> I took some tpstats (roughly every minutes). Only these pools are not at 0
> (Active/Pending).
>
> Pool Name                    Active   Pending      Completed   Blocked
>  All time blocked
> MemtableFlushWriter               2         2         139864         0
>             0
> MemtablePostFlush                 1        13         223714         0
>             0
> CompactionExecutor               10        10         804964         0
>             0
>
> MemtableFlushWriter               4         4         139889         0
>             0
> MemtablePostFlush                 1        12         223744         0
>             0
> CompactionExecutor               12        12         805365         0
>             0
>
> MemtableFlushWriter               5         5         139896         0
>             0
> MemtablePostFlush                 1        10         223755         0
>             0
> CompactionExecutor                9         9         805503         0
>             0
>
> MemtableFlushWriter               4         4         139907         0
>             0
> MemtablePostFlush                 1        13         223762         0
>             0
> CompactionExecutor                9         9         805703         0
>             0
>
>
> MemtableFlushWriter               5         5         139927         0
>             0
> MemtablePostFlush                 1        14         223783         0
>             0
> CompactionExecutor               10        10         805971         0
>             0
>
> MemtableFlushWriter               7         7         139956         0
>             0
> MemtablePostFlush                 1        23         223806         0
>             0
> CompactionExecutor               10        10         806428         0
>             0
>
> nodetool compactionstats shows pending tasks 66
>
> Keyspace: foo
>         Read Count: 6308735
>         Read Latency: 12.132909585836147 ms.
>         Write Count: 15394697
>         Write Latency: 0.09054346675351908 ms.
>         Pending Flushes: 15
>                 Table: bar
>                 SSTable count: 10326
>                 SSTables in each level: [10090/4, 10, 106/100, 112, 0, 0,
> 0, 0, 0]
>                 Space used (live): 69204087872
>                 Space used (total): 69206400092
>                 Space used by snapshots (total): 2708047105
>                 Off heap memory used (total): 35230672
>                 SSTable Compression Ratio: 0.339043411676821
>                 Number of keys (estimate): 1601158
>                 Memtable cell count: 86524
>                 Memtable data size: 6508214
>                 Memtable off heap memory used: 0
>                 Memtable switch count: 22719
>                 Local read count: 6310549
>                 Local read latency: 12.135 ms
>                 Local write count: 15397653
>                 Local write latency: 0.091 ms
>                 Pending flushes: 10
>                 Bloom filter false positives: 2282107
>                 Bloom filter false ratio: 0.38494
>                 Bloom filter space used: 3244792
>                 Bloom filter off heap memory used: 3162168
>                 Index summary off heap memory used: 3348360
>                 Compression metadata off heap memory used: 28720144
>                 Compacted partition minimum bytes: 87
>                 Compacted partition maximum bytes: 2816159
>                 Compacted partition mean bytes: 69860
>                 Average live cells per slice (last five minutes):
> 817.6059838850788
>                 Maximum live cells per slice (last five minutes): 5002.0
>                 Average tombstones per slice (last five minutes): 0.0
>                 Maximum tombstones per slice (last five minutes): 0.0
>
> Thanks
>
> J-F Gosselin
>
>


-- 
Close the World, Open the Net
http://www.linux-wizard.net

--001a113d033e4e81970533493aad
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: base64

PGRpdiBkaXI9Imx0ciI+PGRpdj48ZGl2PjxkaXY+PGRpdj5BcmUgeW91IHVzaW5nIHJlcGFpclBh
cmFsbGVsaXNtID0gc2VxdWVudGlhbCBvciBwYXJhbGxlbCA/PGJyPjxicj48L2Rpdj5BcyBzYWlk
IGJ5IEFsYWluOjxicj48L2Rpdj4tIHRyeSB0byBkZWNyZWFzZSBzdHJlYW10aHJvdWdocHV0IHRv
IGF2b2lkIG92ZXJmbG9vZGluZyBub2RlcyB3aXRoIGEgbG90cyBvZiAoc21hbGwpIHN0cmVhbWVk
IHNzdGFibGVzPGJyPjwvZGl2Pi0gaWYgeW91IGFyZSB1c2luZyAvLyByZXBhaXIsIHN3aXRjaCB0
byBzZXF1ZW50aWFsPGJyPjwvZGl2PjxkaXY+LSBkb24mIzM5O3Qgc3RhcnQgdG9vIG11Y2ggcmVw
YWlyIHNpbXVsdGFuZW91c2x5Ljxicj48L2Rpdj4tIERvIHlvdSByZWFsbHkgbmVlZCB0byB1c2Ug
TENTIGZvciB5b3VyIHRhYmxlcyA/IExDUyBtYWtlIHRoZSBwcm9ibGVtIGV2ZW4gd29yc2UuIFVz
ZSBpdCB3aXRoIHBhcmNpbW9ueSA7KTxicj48YnI+PGRpdj48YnI+PC9kaXY+PC9kaXY+PGRpdiBj
bGFzcz0iZ21haWxfZXh0cmEiPjxicj48ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+MjAxNi0wNS0w
NiAxODowNSBHTVQrMDI6MDAgSmVhbi1GcmFuY29pcyBHb3NzZWxpbiA8c3BhbiBkaXI9Imx0ciI+
Jmx0OzxhIGhyZWY9Im1haWx0bzpqZmdvc3NlbGluQGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsi
PmpmZ29zc2VsaW5AZ21haWwuY29tPC9hPiZndDs8L3NwYW4+Ojxicj48YmxvY2txdW90ZSBjbGFz
cz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MCAwIDAgLjhleDtib3JkZXItbGVmdDoxcHgg
I2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij48ZGl2IHN0eWxlPSJ3b3JkLXdyYXA6YnJlYWst
d29yZCI+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6SGVsdmV0aWNhLEFyaWFsO2ZvbnQtc2l6ZTox
M3B4O2NvbG9yOnJnYmEoMCwwLDAsMS4wKTttYXJnaW46MHB4O2xpbmUtaGVpZ2h0OmF1dG8iPjxz
cGFuIHN0eWxlPSJmb250LWZhbWlseTpoZWx2ZXRpY2EsYXJpYWwiPi0gQ2Fzc2FuZHJhIDIuMS4x
M8KgPC9zcGFuPjwvZGl2PjxkaXY+PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6aGVsdmV0aWNhLGFy
aWFsO2ZvbnQtc2l6ZToxM3B4Ij48ZGl2Pi0gU1NEczwvZGl2PjxkaXY+LSBMZXZlbGVkQ29tcGFj
dGlvblN0cmF0ZWd5IMKgwqA8L2Rpdj48ZGl2Pi0gUmFuZ2UgcmVwYWlyIChub3QgaW5jcmVtZW50
YWwpIHdpdGggU3BvdGlmeSYjMzk7cyBSZWFwZXIgPGEgaHJlZj0iaHR0cHM6Ly9naXRodWIuY29t
L3Nwb3RpZnkvY2Fzc2FuZHJhLXJlYXBlciIgdGFyZ2V0PSJfYmxhbmsiPmh0dHBzOi8vZ2l0aHVi
LmNvbS9zcG90aWZ5L2Nhc3NhbmRyYS1yZWFwZXI8L2E+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRp
dj5Qcm9ibGVtIDogV2hlbiB3ZSBydW4gYSByZXBhaXIgam9iIHNvbWV0aW1lcyB0aGUgU1NUYWJs
ZSBjb3VudCBnb2VzIHRvIDEwSyBvbiBvbmUgb2Ygbm9kZXMgKG5vdCBhbHdheXMgdGhlIHNhbWUg
bm9kZSkuIFRoZSBSZWFwZXIgaXMgc21hcnQgZW5vdWdoIHRvIHBvc3Rwb25lIHRoZSByZXBhaXIg
b24gdGhpcyBub2RlIHNpbmNlIHRoZSBudW1iZXIgb2YgcGVuZGluZyBjb21wYWN0aW9ucyBpcyAm
Z3Q7IDIwIGJ1dCBudW1iZXIgb2YgU1NUYWJsZXMgc3RheXMgYXJvdW5kIDEwSy48L2Rpdj48ZGl2
PkV2ZW4gSWYgSSBzZXQgdGhlIGNvbXBhY3Rpb250aHJvdWdocHV0IDAgKGRpc2FibGUgdGhyb3R0
bGluZykgdGhlIFNTVGFibGUgY291bnQgc3RheXMgYXJvdW5kIDEwSy7CoDwvZGl2PjxkaXY+PGJy
PjwvZGl2PjxkaXY+V29ya2Fyb3VuZDogSWYgd2UgYWJvcnQgdGhlIHJlcGFpciwgYW5kIHJlc3Rh
cnQgdGhlIG5vZGUgaXQgcXVpY2tseSAoaW4gMTUgbWludXRlcykgZ29lcyBiYWNrIHRvIDIwMCBT
U1RhYmxlcyAuLi48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PkFueSBzdWdnZXN0aW9ucyBhcyB3
aGF0IEkgc2hvdWxkIGxvb2sgYXQgPzwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+V2hlbiBpdCBv
Y2N1cnMsIEkmIzM5O3ZlIG5vdGljZWQgdGhhdCBub2RldG9vbCBjb21wYWN0aW9uc3RhdHMgYW5k
IGNmc3RhdHMgKG9uIHRoZSB0YWJsZSB3aXRoIDEwSyBTU1RhYmxlcykgdGFrZXMgbWludXRlcyB0
byByZXR1cm4gd2l0aCBhIHJlc3VsdC48L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PkkgdGhvdWdo
dCB0aGF0IHRoZSBpc3N1ZSBtaWdodCBiZSByZWxhdGVkIHRvIDxhIGhyZWY9Imh0dHBzOi8vaXNz
dWVzLmFwYWNoZS5vcmcvamlyYS9icm93c2UvQ0FTU0FORFJBLTEwNzY2IiB0YXJnZXQ9Il9ibGFu
ayI+aHR0cHM6Ly9pc3N1ZXMuYXBhY2hlLm9yZy9qaXJhL2Jyb3dzZS9DQVNTQU5EUkEtMTA3NjY8
L2E+IGFzIEkgc2VlIHRoZSBNZW10YWJsZVBvc3RGbHVzaCB3YWl0aW5nIG9uIHRoZSBjb3VudGRv
d24gbGF0Y2ggYnV0IHRoZSBQZW5kaW5nIE1lbXRhYmxlUG9zdEZsdXNoIGlzIGdvaW5nIHVwIGFu
ZCBkb3duIGFjY29yZGluZyB0byB0cHN0YXRzLjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+Q29t
cGxldGUgc3RhY2sgdHJhY2UgOsKgPGEgaHJlZj0iaHR0cDovL3Bhc3RlYmluLmNvbS9LMXIzQ1Vm
ZiIgdGFyZ2V0PSJfYmxhbmsiPmh0dHA6Ly9wYXN0ZWJpbi5jb20vSzFyM0NVZmY8L2E+PC9kaXY+
PGRpdj48YnI+PC9kaXY+PGRpdj5JIHRvb2sgc29tZSB0cHN0YXRzIChyb3VnaGx5IGV2ZXJ5IG1p
bnV0ZXMpLiBPbmx5IHRoZXNlIHBvb2xzIGFyZSBub3QgYXQgMCAoQWN0aXZlL1BlbmRpbmcpLsKg
PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5Qb29sIE5hbWUgwqAgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgwqBBY3RpdmUgwqAgUGVuZGluZyDCoCDCoCDCoENvbXBsZXRlZCDCoCBCbG9ja2VkIMKg
QWxsIHRpbWUgYmxvY2tlZDwvZGl2PjxkaXY+TWVtdGFibGVGbHVzaFdyaXRlciDCoCDCoCDCoCDC
oCDCoCDCoCDCoCAyIMKgIMKgIMKgIMKgIDIgwqAgwqAgwqAgwqAgMTM5ODY0IMKgIMKgIMKgIMKg
IDAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgMDwvZGl2PjxkaXY+TWVtdGFibGVQb3N0Rmx1c2gg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgMSDCoCDCoCDCoCDCoDEzIMKgIMKgIMKgIMKgIDIyMzcx
NCDCoCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDA8L2Rpdj48ZGl2PkNvbXBh
Y3Rpb25FeGVjdXRvciDCoCDCoCDCoCDCoCDCoCDCoCDCoCAxMCDCoCDCoCDCoCDCoDEwIMKgIMKg
IMKgIMKgIDgwNDk2NCDCoCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDA8L2Rp
dj48ZGl2Pjxicj48L2Rpdj48ZGl2Pk1lbXRhYmxlRmx1c2hXcml0ZXIgwqAgwqAgwqAgwqAgwqAg
wqAgwqAgNCDCoCDCoCDCoCDCoCA0IMKgIMKgIMKgIMKgIDEzOTg4OSDCoCDCoCDCoCDCoCAwIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIDA8L2Rpdj48ZGl2Pk1lbXRhYmxlUG9zdEZsdXNoIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIDEgwqAgwqAgwqAgwqAxMiDCoCDCoCDCoCDCoCAyMjM3NDQgwqAg
wqAgwqAgwqAgMCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAwPC9kaXY+PGRpdj5Db21wYWN0aW9u
RXhlY3V0b3IgwqAgwqAgwqAgwqAgwqAgwqAgwqAgMTIgwqAgwqAgwqAgwqAxMiDCoCDCoCDCoCDC
oCA4MDUzNjUgwqAgwqAgwqAgwqAgMCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAwPC9kaXY+PGRp
dj48YnI+PC9kaXY+PGRpdj5NZW10YWJsZUZsdXNoV3JpdGVyIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IDUgwqAgwqAgwqAgwqAgNSDCoCDCoCDCoCDCoCAxMzk4OTYgwqAgwqAgwqAgwqAgMCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCAwPC9kaXY+PGRpdj5NZW10YWJsZVBvc3RGbHVzaCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCAxIMKgIMKgIMKgIMKgMTAgwqAgwqAgwqAgwqAgMjIzNzU1IMKgIMKgIMKg
IMKgIDAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgMDwvZGl2PjxkaXY+Q29tcGFjdGlvbkV4ZWN1
dG9yIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgOSDCoCDCoCDCoCDCoCA5IMKgIMKgIMKgIMKgIDgw
NTUwMyDCoCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDA8L2Rpdj48ZGl2Pjxi
cj48L2Rpdj48ZGl2Pk1lbXRhYmxlRmx1c2hXcml0ZXIgwqAgwqAgwqAgwqAgwqAgwqAgwqAgNCDC
oCDCoCDCoCDCoCA0IMKgIMKgIMKgIMKgIDEzOTkwNyDCoCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIDA8L2Rpdj48ZGl2Pk1lbXRhYmxlUG9zdEZsdXNoIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIMKgIDEgwqAgwqAgwqAgwqAxMyDCoCDCoCDCoCDCoCAyMjM3NjIgwqAgwqAgwqAgwqAg
MCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAwPC9kaXY+PGRpdj5Db21wYWN0aW9uRXhlY3V0b3Ig
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqA5IMKgIMKgIMKgIMKgIDkgwqAgwqAgwqAgwqAgODA1NzAz
IMKgIMKgIMKgIMKgIDAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgMDwvZGl2PjxkaXY+PGJyPjwv
ZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+TWVtdGFibGVGbHVzaFdyaXRlciDCoCDCoCDCoCDCoCDC
oCDCoCDCoCA1IMKgIMKgIMKgIMKgIDUgwqAgwqAgwqAgwqAgMTM5OTI3IMKgIMKgIMKgIMKgIDAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgMDwvZGl2PjxkaXY+TWVtdGFibGVQb3N0Rmx1c2ggwqAg
wqAgwqAgwqAgwqAgwqAgwqAgwqAgMSDCoCDCoCDCoCDCoDE0IMKgIMKgIMKgIMKgIDIyMzc4MyDC
oCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDA8L2Rpdj48ZGl2PkNvbXBhY3Rp
b25FeGVjdXRvciDCoCDCoCDCoCDCoCDCoCDCoCDCoCAxMCDCoCDCoCDCoCDCoDEwIMKgIMKgIMKg
IMKgIDgwNTk3MSDCoCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDA8L2Rpdj48
ZGl2Pjxicj48L2Rpdj48ZGl2Pk1lbXRhYmxlRmx1c2hXcml0ZXIgwqAgwqAgwqAgwqAgwqAgwqAg
wqAgNyDCoCDCoCDCoCDCoCA3IMKgIMKgIMKgIMKgIDEzOTk1NiDCoCDCoCDCoCDCoCAwIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIMKgIDA8L2Rpdj48ZGl2Pk1lbXRhYmxlUG9zdEZsdXNoIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIDEgwqAgwqAgwqAgwqAyMyDCoCDCoCDCoCDCoCAyMjM4MDYgwqAgwqAg
wqAgwqAgMCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAwPC9kaXY+PGRpdj5Db21wYWN0aW9uRXhl
Y3V0b3IgwqAgwqAgwqAgwqAgwqAgwqAgwqAgMTAgwqAgwqAgwqAgwqAxMCDCoCDCoCDCoCDCoCA4
MDY0MjggwqAgwqAgwqAgwqAgMCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAwPC9kaXY+PGRpdj48
YnI+PC9kaXY+PGRpdj5ub2RldG9vbCBjb21wYWN0aW9uc3RhdHMgc2hvd3MgcGVuZGluZyB0YXNr
cyA2NjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+S2V5c3BhY2U6IGZvbzwvZGl2PjxkaXY+wqAg
wqAgwqAgwqAgUmVhZCBDb3VudDogNjMwODczNTwvZGl2PjxkaXY+wqAgwqAgwqAgwqAgUmVhZCBM
YXRlbmN5OiAxMi4xMzI5MDk1ODU4MzYxNDcgbXMuPC9kaXY+PGRpdj7CoCDCoCDCoCDCoCBXcml0
ZSBDb3VudDogMTUzOTQ2OTc8L2Rpdj48ZGl2PsKgIMKgIMKgIMKgIFdyaXRlIExhdGVuY3k6IDAu
MDkwNTQzNDY2NzUzNTE5MDggbXMuPC9kaXY+PGRpdj7CoCDCoCDCoCDCoCBQZW5kaW5nIEZsdXNo
ZXM6IDE1PC9kaXY+PGRpdj7CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBUYWJsZTogYmFyPC9kaXY+
PGRpdj7CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBTU1RhYmxlIGNvdW50OiAxMDMyNjwvZGl2Pjxk
aXY+wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgU1NUYWJsZXMgaW4gZWFjaCBsZXZlbDogWzEwMDkw
LzQsIDEwLCAxMDYvMTAwLCAxMTIsIDAsIDAsIDAsIDAsIDBdPC9kaXY+PGRpdj7CoCDCoCDCoCDC
oCDCoCDCoCDCoCDCoCBTcGFjZSB1c2VkIChsaXZlKTogNjkyMDQwODc4NzI8L2Rpdj48ZGl2PsKg
IMKgIMKgIMKgIMKgIMKgIMKgIMKgIFNwYWNlIHVzZWQgKHRvdGFsKTogNjkyMDY0MDAwOTI8L2Rp
dj48ZGl2PsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIFNwYWNlIHVzZWQgYnkgc25hcHNob3RzICh0
b3RhbCk6IDI3MDgwNDcxMDU8L2Rpdj48ZGl2PsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIE9mZiBo
ZWFwIG1lbW9yeSB1c2VkICh0b3RhbCk6IDM1MjMwNjcyPC9kaXY+PGRpdj7CoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCBTU1RhYmxlIENvbXByZXNzaW9uIFJhdGlvOiAwLjMzOTA0MzQxMTY3NjgyMTwv
ZGl2PjxkaXY+wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgTnVtYmVyIG9mIGtleXMgKGVzdGltYXRl
KTogMTYwMTE1ODwvZGl2PjxkaXY+wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgTWVtdGFibGUgY2Vs
bCBjb3VudDogODY1MjQ8L2Rpdj48ZGl2PsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIE1lbXRhYmxl
IGRhdGEgc2l6ZTogNjUwODIxNDwvZGl2PjxkaXY+wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgTWVt
dGFibGUgb2ZmIGhlYXAgbWVtb3J5IHVzZWQ6IDA8L2Rpdj48ZGl2PsKgIMKgIMKgIMKgIMKgIMKg
IMKgIMKgIE1lbXRhYmxlIHN3aXRjaCBjb3VudDogMjI3MTk8L2Rpdj48ZGl2PsKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIExvY2FsIHJlYWQgY291bnQ6IDYzMTA1NDk8L2Rpdj48ZGl2PsKgIMKgIMKg
IMKgIMKgIMKgIMKgIMKgIExvY2FsIHJlYWQgbGF0ZW5jeTogMTIuMTM1IG1zPC9kaXY+PGRpdj7C
oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBMb2NhbCB3cml0ZSBjb3VudDogMTUzOTc2NTM8L2Rpdj48
ZGl2PsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIExvY2FsIHdyaXRlIGxhdGVuY3k6IDAuMDkxIG1z
PC9kaXY+PGRpdj7CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBQZW5kaW5nIGZsdXNoZXM6IDEwPC9k
aXY+PGRpdj7CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBCbG9vbSBmaWx0ZXIgZmFsc2UgcG9zaXRp
dmVzOiAyMjgyMTA3PC9kaXY+PGRpdj7CoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBCbG9vbSBmaWx0
ZXIgZmFsc2UgcmF0aW86IDAuMzg0OTQ8L2Rpdj48ZGl2PsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg
IEJsb29tIGZpbHRlciBzcGFjZSB1c2VkOiAzMjQ0NzkyPC9kaXY+PGRpdj7CoCDCoCDCoCDCoCDC
oCDCoCDCoCDCoCBCbG9vbSBmaWx0ZXIgb2ZmIGhlYXAgbWVtb3J5IHVzZWQ6IDMxNjIxNjg8L2Rp
dj48ZGl2PsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIEluZGV4IHN1bW1hcnkgb2ZmIGhlYXAgbWVt
b3J5IHVzZWQ6IDMzNDgzNjA8L2Rpdj48ZGl2PsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIENvbXBy
ZXNzaW9uIG1ldGFkYXRhIG9mZiBoZWFwIG1lbW9yeSB1c2VkOiAyODcyMDE0NDwvZGl2PjxkaXY+
wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgQ29tcGFjdGVkIHBhcnRpdGlvbiBtaW5pbXVtIGJ5dGVz
OiA4NzwvZGl2PjxkaXY+wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgQ29tcGFjdGVkIHBhcnRpdGlv
biBtYXhpbXVtIGJ5dGVzOiAyODE2MTU5PC9kaXY+PGRpdj7CoCDCoCDCoCDCoCDCoCDCoCDCoCDC
oCBDb21wYWN0ZWQgcGFydGl0aW9uIG1lYW4gYnl0ZXM6IDY5ODYwPC9kaXY+PGRpdj7CoCDCoCDC
oCDCoCDCoCDCoCDCoCDCoCBBdmVyYWdlIGxpdmUgY2VsbHMgcGVyIHNsaWNlIChsYXN0IGZpdmUg
bWludXRlcyk6IDgxNy42MDU5ODM4ODUwNzg4PC9kaXY+PGRpdj7CoCDCoCDCoCDCoCDCoCDCoCDC
oCDCoCBNYXhpbXVtIGxpdmUgY2VsbHMgcGVyIHNsaWNlIChsYXN0IGZpdmUgbWludXRlcyk6IDUw
MDIuMDwvZGl2PjxkaXY+wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgQXZlcmFnZSB0b21ic3RvbmVz
IHBlciBzbGljZSAobGFzdCBmaXZlIG1pbnV0ZXMpOiAwLjA8L2Rpdj48ZGl2PsKgIMKgIMKgIMKg
IMKgIMKgIMKgIMKgIE1heGltdW0gdG9tYnN0b25lcyBwZXIgc2xpY2UgKGxhc3QgZml2ZSBtaW51
dGVzKTogMC4wPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5UaGFua3M8L2Rpdj48ZGl2Pjxicj48
L2Rpdj48ZGl2PkotRiBHb3NzZWxpbjwvZGl2PjxkaXY+PGJyPjwvZGl2PjwvZGl2PjwvZGl2Pjwv
ZGl2PjwvYmxvY2txdW90ZT48L2Rpdj48YnI+PGJyIGNsZWFyPSJhbGwiPjxicj4tLSA8YnI+PGRp
diBjbGFzcz0iZ21haWxfc2lnbmF0dXJlIj5DbG9zZSB0aGUgV29ybGQsIE9wZW4gdGhlIE5ldDxi
cj48YSBocmVmPSJodHRwOi8vd3d3LmxpbnV4LXdpemFyZC5uZXQiIHRhcmdldD0iX2JsYW5rIj5o
dHRwOi8vd3d3LmxpbnV4LXdpemFyZC5uZXQ8L2E+PC9kaXY+DQo8L2Rpdj4NCg==
--001a113d033e4e81970533493aad--