Mailing-List: contact users-help@tomcat.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "Tomcat Users List" <users@tomcat.apache.org>
Received-SPF: neutral (nike.apache.org: local policy)
Message-ID: <504923B6.5060107@christopherschultz.net>
Date: Thu, 06 Sep 2012 18:29:10 -0400
From: Christopher Schultz <chris@christopherschultz.net>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8;
 rv:15.0) Gecko/20120824 Thunderbird/15.0
MIME-Version: 1.0
To: Tomcat Users List <users@tomcat.apache.org>
Subject: Re: Tuning session replication on clusters
References: 
 <OFB4EE61A7.4AC4E5A3-ON86257A70.006C193C-86257A70.006D0A9E@LocalDomain>
 <CAAKASGyw18-E2QVOd2gvJN1QFtxGPuRSTDMvR=d2YQssSW2O4Q@mail.gmail.com>
 <OF38912B89.B362E506-ON86257A71.00026F1F-86257A71.00046CAE@LocalDomain>
 <OF6BACFBFC.92FA4B5A-ON86257A71.0009FE1E-86257A71.000AFA98@LocalDomain>
 <5048C319.2080709@christopherschultz.net>
 <OFE65C5946.2080F4ED-ON86257A71.0062DAD6-86257A71.00671B38@LocalDomain>
In-Reply-To: 
 <OFE65C5946.2080F4ED-ON86257A71.0062DAD6-86257A71.00671B38@LocalDomain>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Kyle,

On 9/6/12 2:45 PM, kharper2@oreillyauto.com wrote:
> Chris:
>> One question: Plot those as X-Y in a spreadsheet and you'll see
>> that it's only a bit worse than linear, especially after 1500.
>> There's no enough data presented to draw an "exponential
>> performance curve" conclusion: you're going to need more data to
>> see if this is worse than linear. Can you collect more data? I
>> know that creating 42k sessions of 700MiB each is probably a ..
>> challenge, but more data would certainly be helpful.
> 
> Before I change anything (tuning) I'll do as much testing as I can
> tonight to generate a data series and graph.  Btw, does this list
> support .ods/.xls attachments?

I'm not sure. You might be better off wrapping the file in a ZIP file
(which is silly, since OpenDocument is already ZIP, but it might help)
or just throwing the file up on pastebin (which I generally don't like
because the message goes into the archives but pastebin items expire,
etc. Just try not to drop a multi-MiB file onto the list ;)

> I'm not sure how Tomcat handles building the object containing all
> the sessions to send (Collection/List?), but it'll be interesting
> to see if there are points where the basic nature of the Collection
> object becomes less efficient.  The testing so far has already 
> shown that the quantity of sessions (like the 42k test I did in 901
> ms on a different app) doesn't appear to be an issue, while large
> session sizes do.

Assembling the sessions into a Collection is likely to be very fast,
since it's just copying references around: the size of the individual
sessions should not matter. Of course, pushing all those bytes to the
other servers...

> I know that the JVM was forced to make a full copy of all the
> session data for the application inside heap space too, so after
> certain points I might be reaching contention with memory
> management processes that handle heap size, ratios, garbage
> collection, etc.

Perhaps Tomcat does something like serialize the session to a big
binary structure and then sends that (which sounds insane -- streaming
binary data is how that should be done -- but I haven't checked to
code to be sure).

> Having seen how quickly the JVM ramped up memory usage for
> replication, I'm relatively confident the session aggregation and
> duplication into a new Collection on the sender side is happening
> fast... reallllly fast.  I'll try to get a protocol analyzer in
> place to capture the packets and compare them to a basic file
> transfer of a single large file.

Sounds good.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlBJI7YACgkQ9CaO5/Lv0PDcIwCfYaKt6ddWk3Nok6bxR2RX8ouX
XEYAni+8N/oGPx/vl9GTwPfVXEek0u/Y
=Tdtz
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org