Return-Path: 
 <user-return-8779-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org>
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: (qmail 945 invoked from network); 27 Aug 2010 04:43:01 -0000
Received: from unknown (HELO mail.apache.org) (140.211.11.3)
  by 140.211.11.9 with SMTP; 27 Aug 2010 04:43:01 -0000
Received: (qmail 94830 invoked by uid 500); 27 Aug 2010 04:42:59 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 94578 invoked by uid 500); 27 Aug 2010 04:42:57 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@cassandra.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@cassandra.apache.org>
List-Post: <mailto:user@cassandra.apache.org>
List-Id: <user.cassandra.apache.org>
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 94567 invoked by uid 99); 27 Aug 2010 04:42:56 -0000
Received: from Unknown (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Aug 2010 04:42:56 +0000
X-ASF-Spam-Status: No, hits=2.2 required=10.0
	tests=FREEMAIL_FROM,HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of ee.zelo@gmail.com designates
 209.85.212.44 as permitted sender)
Received: from [209.85.212.44] (HELO mail-vw0-f44.google.com) (209.85.212.44)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Aug 2010 04:42:33 +0000
Received: by vws10 with SMTP id 10so2744079vws.31
        for <user@cassandra.apache.org>; Thu, 26 Aug 2010 21:42:12 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:mime-version:received:received:in-reply-to
         :references:date:message-id:subject:from:to:content-type;
        bh=fr0s5hrropbzSmBNCWKfgpDv5655iXyyN+arjYSrL+c=;
        b=U44TGezC/AEj2aJXVlCUQdhG7r+Ph5MKZxCjqFmZrJELlxrjUJtVee2P32z/n3hy9i
         yh33HIJeb8fsmUhgGr2jcTpKIftDsgX521CShZBzOix8EcemmBCB+Fz4x5CyJNBz/fSK
         liV+H6o0WWjN2RGMXB8zpSnRB/9dbETNtZTfk=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=pq19Jcyp+bnh3b+PpPaOZ7v2WPgopEQuMwhm4VRZ6pNyKjnMCmjykRoeBs6d2SySoY
         0V2z7q4v/UNfTVtA7w+/fx9y/JEOzTgRWArs3IZUXqDZS6Jp7PZAUROCzzsAmi2+HwlR
         vbTdGmgNlnL71uXzKFtypVK6RIwcMAxGDdABU=
MIME-Version: 1.0
Received: by 10.220.171.211 with SMTP id i19mr165452vcz.112.1282884131966;
 Thu, 26 Aug 2010 21:42:11 -0700 (PDT)
Received: by 10.220.190.9 with HTTP; Thu, 26 Aug 2010 21:42:11 -0700 (PDT)
In-Reply-To: <AANLkTimALBCbLfnb_POSzJ=LK=6Yn95kkbwkH2ZLwiZL@mail.gmail.com>
References: <AANLkTimALBCbLfnb_POSzJ=LK=6Yn95kkbwkH2ZLwiZL@mail.gmail.com>
Date: Fri, 27 Aug 2010 00:42:11 -0400
Message-ID: <AANLkTinUgC+c-6zMP0ZhcYbvx6pxhn9Qto3fZgerC2C0@mail.gmail.com>
Subject: Re: Ordered Partitioner load balance problem
From: Edward Evans <ee.zelo@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=0016e64cad1ef94f6a048ec6bc73
X-Virus-Checked: Checked by ClamAV on apache.org

--0016e64cad1ef94f6a048ec6bc73
Content-Type: text/plain; charset=ISO-8859-1

Upgraded to 0.6.4 and still seeing this behavior. Any help is appreciated.

On Thu, Aug 26, 2010 at 1:03 AM, Edward Evans <ee.zelo@gmail.com> wrote:

> I am currently using Cassandra 0.6.2 on four virtual nodes in two different
> data centers (A, B). My initial testing used the Random Partitioner and
> everything behaved as expected. I moved to the Ordered Partitioner using
> SHA256 hashes as the keys and subsequently these are the tokens (If the
> stories I am told are true). My hope is that, in defining the initial tokens
> correctly, I would see random and even load balancing.
>
> My keyspace is defined rather simply as:
>
> <Keyspace Name="cc-count5">
>         <ColumnFamily Name="StandardCount"/>
>
>
>  <ReplicaPlacementStrategy>org.apache.cassandra.locator.RackUnawareStrategy</ReplicaPlacementStrategy>
>       <!-- Number of replicas of the data -->
>       <ReplicationFactor>2</ReplicationFactor>
>
>
>  <EndPointSnitch>org.apache.cassandra.locator.EndPointSnitch</EndPointSnitch>
> </Keyspace>
>
> Since in 0.6.2, comparison is a utf8 string comparison I define my initial
> tokens as follows. (note: 3rd octet of Address represents the DC)
>
> Address       Status     Load          Range
>        Ring
>
> ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
> 192.168.152.237Up         0 bytes
> 3fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff|<--|
> 192.168.136.179Up         0 bytes
> 7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff|   |
> 192.168.152.238Up         0 bytes
> bfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff|   |
> 192.168.146.254Up         0 bytes
> ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff|-->|
>
> using clustertool to confirm this could be correct:
>
> [ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints
> cc-count5 a213c14de9c1d4464aedd84eff70ae91b83b6937d11feaeb02d25f36a622d05c
> Key              :
> a213c14de9c1d4464aedd84eff70ae91b83b6937d11feaeb02d25f36a622d05c
> Endpoints        : [/192.168.152.238, /192.168.146.254]
> [ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints
> cc-count5 1ad31e4f9ed64d9d056ceb9363c8ceeb7c8e65fb0931d43f85dac7aff9152f43
> Key              :
> 1ad31e4f9ed64d9d056ceb9363c8ceeb7c8e65fb0931d43f85dac7aff9152f43
> Endpoints        : [/192.168.152.237, /192.168.136.179]
> [ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints
> cc-count5 e71ca1e6af17d22c40c93bd3e9814f66dd371d00e366988a5d44701dc809bd45
> Key              :
> e71ca1e6af17d22c40c93bd3e9814f66dd371d00e366988a5d44701dc809bd45
> Endpoints        : [/192.168.146.254, /192.168.152.237]
> [ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints
> cc-count5 7becffa44913d0b4d5200b763c88dc9251f3796586f160dc260482a6777ea696
> Key              :
> 7becffa44913d0b4d5200b763c88dc9251f3796586f160dc260482a6777ea696
> Endpoints        : [/192.168.136.179, /192.168.152.238]
> [ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints
> cc-count5 7ffffff44913d0b4d5200b763c88dc9251f3796586f160dc260482a6777ea696
> Key              :
> 7ffffff44913d0b4d5200b763c88dc9251f3796586f160dc260482a6777ea696
> Endpoints        : [/192.168.136.179, /192.168.152.238]
> [ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints
> cc-count5 fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe
> Key              :
> fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe
> Endpoints        : [/192.168.146.254, /192.168.152.237]
>
> Looks good. Now doing some (~5M) inserts.....
>
> 192.168.152.237Up         1.22 GB
> 3fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff|<--|
> 192.168.136.179Up         2.16 GB
> 7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff|   |
> 192.168.152.238Up         1.23 GB
> bfffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff|   |
> 192.168.146.254Up         148.97 MB
> ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff|-->|
>
> definitely out of balance.
>
> Some command line perl shows the tokens are evenly distributed through the
> token space, unfortunately they are not going to the correct nodes. This is
> a count of keys from the commit-log on host  192.168.152.237.
> ^1 77068
> ^2 76977
> ^3 77280
> ^4 77065
> ^5 77038
> ^6 76728
> ^7 76976
> ^8 76824
> ^9 76792
> ^a 76794
> ^b 77178
> ^c 76917
> ^d 77074
> ^e 76921
> ^f 76848
>
> Finanlly, Looking at StandardCount-8-Index.db (again on 192.168.152.237) I
> see keys that I *think* should not exist on this host. Also see these is the
> Data file itself.
>
> H2600704.07cb1d3ba32940d82d168c4e8ad30511927752f3d66dba369bb4377ddec31e40
> H2600705.070b88ff8bad566a92f6090e678cc2ed734d29cdf2cfb1fbf4ef69cd45968b12
> H2600706.8a53bbc1725f81d0dcf945f29646352ad61a7092f4d1476f03c52f73cf9d0964
> H2600707.6938f1810c443e4007c65e97997b64c886a83788f9fae7a72339fd0df702dd0d
> H2600708.a8cfa25e72d68edfe2a552a34145f403d5d0b977ff245b053b2f1fef9aebf3e3
> H2600709.11513c343acc885f0f315e8c0d05a71c3f5369d2339ad2860740dfe416a129d8
> H2600710.bcc4519c8c20d5ac1565752a749500b2138982c1595bed0c19d6582f6d1f364b
> H2600711.a55a2077cb887d9510645d4814e37b70341981a8b112474b179ea6a20ce4d061
> H2600712.c0be35e21309c5138c623033f5e2d795475fdcb19d6fcf925d66a892bba3e511
> H2600713.62d8ad66d5bc65f597ac469089e252d3034777553fb3fc9ea3b966fa6a1a5922
> H2600714.c7af0277f8e2aea7d26701648708d1698dde849af547af216d5bdeab63acec60
> H2600715.fe3bab952d2cae53bb06e9c2ef8abcbf5f08bf02009422d55e82b58f04ecdcb0
> H2600716.9cd05c7f6b806d34498eac313d375431a00dce3cc64e4d1a18827687a44ca99a
> H2600717.086f36abf2b5bc9234a89028ffcccd1607ba1253acf46d8c61e46c3e392438ff
> H2600718.fbacc6bd2a6572cb3b8b2c1f5b8ce9e99cc619b9d76695e4c3cb0c01be2a4bc8
> H2600719.69d0b298a89b85394c328ca87c12577adb2f7e289bc50b2eb03885b3b7641f00
> H2600720.c52e0c1ec88836d45fcee9a0f65539e7b9a9a9ef29ad7e3befa1add81b0822a9
> H2600721.74e776b48ae66fc4ebbb68dbd94c7470de1e39dca508538866747d8a7a55815d
> H2600722.6f8b1bd5c70ab8b3a2716b980f1ffe0885444d98a9b748aca03625709904e1ef
> H2600723.a14f4a929b83782a3643122981f5eacd1e36ce48c95cd550ad4b068215b93d93
> H2600724.6463a74790168c31b3f0a4edf71487f67df36026ee7bda552ba74b4e60afc119
> H2600725.7f87d7980f636cff5904f46ce86f8b530cdd745b13cd0e1aba9b76980ae6fda4
> H2600726.2ab8636baf10a6da9d232033c0dbd060d4ae208ec39db2e029a73694d051453c
> H2600727.91cf1ae3e787f569756ae8ec12766cdd3a5dc3454b232c8f42df41aa5615bead
> H2600728.71846f16f1b21b002226e65b4ed072377bc187762611ad8e6a09aee57461a48c
> H2600729.598f2c23e9262fa141fee951cadcdde67396c166845d110526f0f210c6519edc
> H2600730.e9fa44e98b30758b0f7af98169e14774fbfa2b017efd7c5425d40a662f92b0cd
> H2600731.00e7562b73f924259f96e39454aff9801806dcb7dabc790501a7fb558dcad55e
> H2600732.a7a9288c6258fb1a538059dddd773cca3db4827b74a8e5d6d075a0035126e2d0
> H2600733.b675424304c639925ebf1afa501d6fdfedcb418a0b609aba21284edc8cf90fd3
> H2600734.296b6da84fdf575736c004d69e2d7d4f4d0cc1caf9a31e53402f8753cdf07015
> H2600735.3de5a1a34ebae67f7981cd0e294a24e11f056cb8fd66226136cfa933e356d26d
> H2600736.032d351b7d596584de11f4499f85f732a1b38a9eee9dcc2b0df4896b10e67ab8
> H2600737.5b5015c97e780bf8ca073a660af913aa524ae1122d8a5ae14ce26f74a8b43b65
> H2600738.73d14b7553b2f82c7d06d3a832c1307a9b0cd7bf3f24ec1bcc389b010d7d52fd
> H2600739.b7166b6fd2c85eae52760e26957c1fccbdc8c7ed61dffa244966ef18572185ca
> H2600740.dff5e7342254303453d22675c363c43a7bc871c6580d6eb81380014ee4e2223b
> H2600741.f07cbdfb65a31182aacdcb64b1c32d8f8909bf47b6a56d6a4cf14e8e23536be7
> H2600742.996389925b25c66e9aa4d8e807cdc403d7853787e55eed83ff66e35c62d22b2d
>
> What am I doing wrong here? Thanks.
>
> -ee-
>

--0016e64cad1ef94f6a048ec6bc73
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Upgraded to 0.6.4 and still seeing this behavior. Any help is appreciated.=
=A0<br><br><div class=3D"gmail_quote">On Thu, Aug 26, 2010 at 1:03 AM, Edwa=
rd Evans <span dir=3D"ltr">&lt;<a href=3D"mailto:ee.zelo@gmail.com">ee.zelo=
@gmail.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;">I am currently using Cassandra=A00.6.2 on f=
our virtual nodes in two different data centers (A, B). My initial testing =
used the Random Partitioner and everything behaved as expected. I moved to =
the Ordered Partitioner using SHA256 hashes as the keys and subsequently th=
ese are the tokens (If the stories I am told are true). My hope is that, in=
 defining the initial tokens correctly, I would see random and even load ba=
lancing.=A0<br>


<br><div>My keyspace is defined rather simply as:=A0<div><br>&lt;Keyspace N=
ame=3D&quot;cc-count5&quot;&gt;<br>=A0=A0 =A0 =A0 =A0&lt;ColumnFamily Name=
=3D&quot;StandardCount&quot;/&gt;<br>=A0=A0 =A0 =A0<br>=A0=A0 =A0 =A0&lt;Re=
plicaPlacementStrategy&gt;org.apache.cassandra.locator.RackUnawareStrategy&=
lt;/ReplicaPlacementStrategy&gt;<br>


=A0=A0 =A0 =A0&lt;!-- Number of replicas of the data --&gt;<br>=A0=A0 =A0 =
=A0&lt;ReplicationFactor&gt;2&lt;/ReplicationFactor&gt;<br>=A0=A0 =A0 =A0<b=
r>=A0=A0 =A0 =A0&lt;EndPointSnitch&gt;org.apache.cassandra.locator.EndPoint=
Snitch&lt;/EndPointSnitch&gt;<br>


&lt;/Keyspace&gt;</div><div><br>Since in 0.6.2,=A0comparison is a utf8 stri=
ng comparison I define my initial tokens as follows. (note: 3rd octet of Ad=
dress represents the DC)<br><br>Address =A0 =A0 =A0 Status =A0 =A0 Load =A0=
 =A0 =A0 =A0 =A0Range =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0Ring<br>


=A0=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff<br>192=
.168.152.237Up =A0 =A0 =A0 =A0 0 bytes =A0 =A0 =A0 3fffffffffffffffffffffff=
ffffffffffffffffffffffffffffffffffffffff|&lt;--|<br>


192.168.136.179Up =A0 =A0 =A0 =A0 0 bytes =A0 =A0 =A0 7ffffffffffffffffffff=
fffffffffffffffffffffffffffffffffffffffffff| =A0 |<br>192.168.152.238Up =A0=
 =A0 =A0 =A0 0 bytes =A0 =A0 =A0 bfffffffffffffffffffffffffffffffffffffffff=
ffffffffffffffffffffff| =A0 |<br>


192.168.146.254Up =A0 =A0 =A0 =A0 0 bytes =A0 =A0 =A0 fffffffffffffffffffff=
fffffffffffffffffffffffffffffffffffffffffff|--&gt;|</div><div><br>using clu=
stertool to confirm this could be correct:<br><br></div><div>[ee@priam cass=
andra]$ ./bin/clustertool -h localhost get_endpoints cc-count5 a213c14de9c1=
d4464aedd84eff70ae91b83b6937d11feaeb02d25f36a622d05c<br>


Key =A0 =A0 =A0 =A0 =A0 =A0 =A0: a213c14de9c1d4464aedd84eff70ae91b83b6937d1=
1feaeb02d25f36a622d05c<br>Endpoints =A0 =A0 =A0 =A0: [/<a href=3D"http://19=
2.168.152.238" target=3D"_blank">192.168.152.238</a>, /<a href=3D"http://19=
2.168.146.254" target=3D"_blank">192.168.146.254</a>]<br>

[ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints cc-count=
5 1ad31e4f9ed64d9d056ceb9363c8ceeb7c8e65fb0931d43f85dac7aff9152f43<br>
Key =A0 =A0 =A0 =A0 =A0 =A0 =A0: 1ad31e4f9ed64d9d056ceb9363c8ceeb7c8e65fb09=
31d43f85dac7aff9152f43<br>Endpoints =A0 =A0 =A0 =A0: [/<a href=3D"http://19=
2.168.152.237" target=3D"_blank">192.168.152.237</a>, /<a href=3D"http://19=
2.168.136.179" target=3D"_blank">192.168.136.179</a>]<br>

[ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints cc-count=
5 e71ca1e6af17d22c40c93bd3e9814f66dd371d00e366988a5d44701dc809bd45<br>
Key =A0 =A0 =A0 =A0 =A0 =A0 =A0: e71ca1e6af17d22c40c93bd3e9814f66dd371d00e3=
66988a5d44701dc809bd45<br>Endpoints =A0 =A0 =A0 =A0: [/<a href=3D"http://19=
2.168.146.254" target=3D"_blank">192.168.146.254</a>, /<a href=3D"http://19=
2.168.152.237" target=3D"_blank">192.168.152.237</a>]<br>

[ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints cc-count=
5 7becffa44913d0b4d5200b763c88dc9251f3796586f160dc260482a6777ea696<br>
Key =A0 =A0 =A0 =A0 =A0 =A0 =A0: 7becffa44913d0b4d5200b763c88dc9251f3796586=
f160dc260482a6777ea696<br>Endpoints =A0 =A0 =A0 =A0: [/<a href=3D"http://19=
2.168.136.179" target=3D"_blank">192.168.136.179</a>, /<a href=3D"http://19=
2.168.152.238" target=3D"_blank">192.168.152.238</a>]<br>

[ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints cc-count=
5 7ffffff44913d0b4d5200b763c88dc9251f3796586f160dc260482a6777ea696<br>
Key =A0 =A0 =A0 =A0 =A0 =A0 =A0: 7ffffff44913d0b4d5200b763c88dc9251f3796586=
f160dc260482a6777ea696<br>Endpoints =A0 =A0 =A0 =A0: [/<a href=3D"http://19=
2.168.136.179" target=3D"_blank">192.168.136.179</a>, /<a href=3D"http://19=
2.168.152.238" target=3D"_blank">192.168.152.238</a>]<br>

[ee@priam cassandra]$ ./bin/clustertool -h localhost get_endpoints cc-count=
5 fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe<br>
Key =A0 =A0 =A0 =A0 =A0 =A0 =A0: ffffffffffffffffffffffffffffffffffffffffff=
fffffffffffffffffffffe<br>Endpoints =A0 =A0 =A0 =A0: [/<a href=3D"http://19=
2.168.146.254" target=3D"_blank">192.168.146.254</a>, /<a href=3D"http://19=
2.168.152.237" target=3D"_blank">192.168.152.237</a>]</div>


<div><br>Looks good. Now doing some (~5M) inserts.....=A0<br><br></div><div=
>192.168.152.237Up =A0 =A0 =A0 =A0 1.22 GB =A0 =A0 =A0 3fffffffffffffffffff=
ffffffffffffffffffffffffffffffffffffffffffff|&lt;--|<br>192.168.136.179Up =
=A0 =A0 =A0 =A0 2.16 GB =A0 =A0 =A0 7ffffffffffffffffffffffffffffffffffffff=
fffffffffffffffffffffffff| =A0 |<br>


192.168.152.238Up =A0 =A0 =A0 =A0 1.23 GB =A0 =A0 =A0 bffffffffffffffffffff=
fffffffffffffffffffffffffffffffffffffffffff| =A0 |<br>192.168.146.254Up =A0=
 =A0 =A0 =A0 148.97 MB =A0 =A0 ffffffffffffffffffffffffffffffffffffffffffff=
ffffffffffffffffffff|--&gt;|<br>


<br></div><div>definitely out of balance.=A0<br></div><div><br></div><div>S=
ome command line perl shows the tokens are evenly distributed through the t=
oken space, unfortunately they are not going to the correct nodes. This is =
a count of keys from the commit-log on host=A0=A0192.168.152.237.=A0<br>


^1 77068<br>^2 76977<br>^3 77280<br>^4 77065<br>^5=A077038<br>^6=A076728<br=
>^7=A076976<br>^8=A076824<br>^9=A076792<br>^a=A076794<br>^b=A077178<br>^c=
=A076917<br>^d=A077074<br>^e=A076921<br>^f 76848<br>=A0<br>Finanlly, Lookin=
g at=A0StandardCount-8-Index.db (again on=A0192.168.152.237) I see keys tha=
t I *think* should not exist on this host. Also see these is the Data file =
itself. =A0<br>


<br>H2600704.07cb1d3ba32940d82d168c4e8ad30511927752f3d66dba369bb4377ddec31e=
40<br>H2600705.070b88ff8bad566a92f6090e678cc2ed734d29cdf2cfb1fbf4ef69cd4596=
8b12<br>H2600706.8a53bbc1725f81d0dcf945f29646352ad61a7092f4d1476f03c52f73cf=
9d0964<br>


H2600707.6938f1810c443e4007c65e97997b64c886a83788f9fae7a72339fd0df702dd0d<b=
r>H2600708.a8cfa25e72d68edfe2a552a34145f403d5d0b977ff245b053b2f1fef9aebf3e3=
<br>H2600709.11513c343acc885f0f315e8c0d05a71c3f5369d2339ad2860740dfe416a129=
d8<br>


H2600710.bcc4519c8c20d5ac1565752a749500b2138982c1595bed0c19d6582f6d1f364b<b=
r>H2600711.a55a2077cb887d9510645d4814e37b70341981a8b112474b179ea6a20ce4d061=
<br>H2600712.c0be35e21309c5138c623033f5e2d795475fdcb19d6fcf925d66a892bba3e5=
11<br>


H2600713.62d8ad66d5bc65f597ac469089e252d3034777553fb3fc9ea3b966fa6a1a5922<b=
r>H2600714.c7af0277f8e2aea7d26701648708d1698dde849af547af216d5bdeab63acec60=
<br>H2600715.fe3bab952d2cae53bb06e9c2ef8abcbf5f08bf02009422d55e82b58f04ecdc=
b0<br>


H2600716.9cd05c7f6b806d34498eac313d375431a00dce3cc64e4d1a18827687a44ca99a<b=
r>H2600717.086f36abf2b5bc9234a89028ffcccd1607ba1253acf46d8c61e46c3e392438ff=
<br>H2600718.fbacc6bd2a6572cb3b8b2c1f5b8ce9e99cc619b9d76695e4c3cb0c01be2a4b=
c8<br>


H2600719.69d0b298a89b85394c328ca87c12577adb2f7e289bc50b2eb03885b3b7641f00<b=
r>H2600720.c52e0c1ec88836d45fcee9a0f65539e7b9a9a9ef29ad7e3befa1add81b0822a9=
<br>H2600721.74e776b48ae66fc4ebbb68dbd94c7470de1e39dca508538866747d8a7a5581=
5d<br>


H2600722.6f8b1bd5c70ab8b3a2716b980f1ffe0885444d98a9b748aca03625709904e1ef<b=
r>H2600723.a14f4a929b83782a3643122981f5eacd1e36ce48c95cd550ad4b068215b93d93=
<br>H2600724.6463a74790168c31b3f0a4edf71487f67df36026ee7bda552ba74b4e60afc1=
19<br>


H2600725.7f87d7980f636cff5904f46ce86f8b530cdd745b13cd0e1aba9b76980ae6fda4<b=
r>H2600726.2ab8636baf10a6da9d232033c0dbd060d4ae208ec39db2e029a73694d051453c=
<br>H2600727.91cf1ae3e787f569756ae8ec12766cdd3a5dc3454b232c8f42df41aa5615be=
ad<br>


H2600728.71846f16f1b21b002226e65b4ed072377bc187762611ad8e6a09aee57461a48c<b=
r>H2600729.598f2c23e9262fa141fee951cadcdde67396c166845d110526f0f210c6519edc=
<br>H2600730.e9fa44e98b30758b0f7af98169e14774fbfa2b017efd7c5425d40a662f92b0=
cd<br>


H2600731.00e7562b73f924259f96e39454aff9801806dcb7dabc790501a7fb558dcad55e<b=
r>H2600732.a7a9288c6258fb1a538059dddd773cca3db4827b74a8e5d6d075a0035126e2d0=
<br>H2600733.b675424304c639925ebf1afa501d6fdfedcb418a0b609aba21284edc8cf90f=
d3<br>


H2600734.296b6da84fdf575736c004d69e2d7d4f4d0cc1caf9a31e53402f8753cdf07015<b=
r>H2600735.3de5a1a34ebae67f7981cd0e294a24e11f056cb8fd66226136cfa933e356d26d=
<br>H2600736.032d351b7d596584de11f4499f85f732a1b38a9eee9dcc2b0df4896b10e67a=
b8<br>


H2600737.5b5015c97e780bf8ca073a660af913aa524ae1122d8a5ae14ce26f74a8b43b65<b=
r>H2600738.73d14b7553b2f82c7d06d3a832c1307a9b0cd7bf3f24ec1bcc389b010d7d52fd=
<br>H2600739.b7166b6fd2c85eae52760e26957c1fccbdc8c7ed61dffa244966ef18572185=
ca<br>


H2600740.dff5e7342254303453d22675c363c43a7bc871c6580d6eb81380014ee4e2223b<b=
r>H2600741.f07cbdfb65a31182aacdcb64b1c32d8f8909bf47b6a56d6a4cf14e8e23536be7=
<br>H2600742.996389925b25c66e9aa4d8e807cdc403d7853787e55eed83ff66e35c62d22b=
2d<br>


<br>What am I doing wrong here? Thanks.=A0</div></div><div><br></div><font =
color=3D"#888888"><div>-ee-</div>
</font></blockquote></div><br>

--0016e64cad1ef94f6a048ec6bc73--