Mailing-List: contact dev-help@myfaces.apache.org; run by ezmlm
Precedence: bulk
Reply-To: "MyFaces Development" <dev@myfaces.apache.org>
Received-SPF: pass (nike.apache.org: domain of gcjmu-myfaces-dev@m.gmane.org
 designates 80.91.229.12 as permitted sender)
To: dev@myfaces.apache.org
From: Werner Punz <werner.punz@gmail.com>
Subject: Re: Web Framework Performance Comparision
Date: Fri, 14 Oct 2011 01:21:59 +0200
Lines: 190
Message-ID: <j77rqo$63i$1@dough.gmane.org>
References: 
 <CA+uAsT4M4n9cEL-C6xwwxLZa5TqaHNEEUfVdmU7W1-MfKEv4dA@mail.gmail.com>
 <j77jlr$iuo$1@dough.gmane.org>
 <CABujnVcC30VG9f8OOHOs4YT8eaqY_Yhw_tN5o9Goskqe3jCudA@mail.gmail.com>
 <j77kmf$q4h$1@dough.gmane.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:7.0.1) Gecko/20110929 Thunderbird/7.0.1
In-Reply-To: <j77kmf$q4h$1@dough.gmane.org>

Actually now that I have some tests at hand, I just
coded some profiling code in.

Here are some numbers for tests which are running on both impls.
The tests have been performed at the Production stage on both impls.
The testing environments where myfaces 2.1.4-SNAPSHOT against Mojarra 
2.1.2. Both running on the same testing framework. The tests usually are 
done from the begin of the testgroup til the end they count more or less 
server and client performance and testing time. But the differences give 
a good indication about the corresponding speeds.

The machine which ran the tests was a macbook pro core2duo with 8 gig 
ram and the tests were run on an ssd. So IO impact is down to a bare 
minimum here. The tests themselves do not do any excessive disk io.


JSF Protocol Tests
MyFaces:  Time for test in ms: 239
Mojarra: Time for test in ms: 207
This is a servlet based test

Testgroup: Full Body Replacement
MyFaces:Time for test in ms: 42
Mojarra:Time for test in ms: 35
This is a servlet based test

Testgroup: Multi form situation
MyFaces: Time for test in ms: 4134
Mojarra:Time for test in ms: 4426
This is a jsf related test

Testgroup: Testing for decorated api calls
MyFaces: Time for test in ms: 24
Mojarra: Time for test in ms: 34

Testgroup: Double Eval Detection:
MyFaces: Time for test in ms: 2047
Mojarra: Time for test in ms: 2060

Testgroup: Table Test, replacement of table elements
MyFaces: Time for test in ms: 7289
Mojarra: Time for test in ms: 9229

Testgroup: Partial Page Rendering Nav Case
MyFaces: Time for test in ms: 2168
Mojarra: Time for test in ms: 2308

Testgroup: MyFaces Event Lifecycle test
MyFaces: Time for test in ms: 130
Mojarra: Time for test in ms: 131

So what we have here is that in most cases especially in all jsf related 
tests myfaces is a little bit faster than Mojarra.
Our scripts seem to be roughly 10% slower, however.But i also still have 
some self induced overhead in the test framework which might count for 
roughly 2-3% which is not active in Mojarra and we deal with corner 
cases mojarra does not cover.

The difference probably is on IE6 even more, because I am dealing more
cleanly with the broken garbage collector of IE6 in my code which causes 
also some performance overhead.

Also have in mind, we deal with more functonality and more code and also 
we have some fixes in which Mojarra yet has to add.
Under normal JSF circumstances by now the server seems to make up for 
the speed difference (normally 95% of the time is spent on the server 
anyway) by being roughly 10-15% faster than Mojarra by now.

If we turn to development the picture turns a little bit with myfaces 
being slower in some tests (I will give the performance numbers tomorrow).

So my conclusion is that they probably have tested against an old Impl 
here.

I will commit the code with the profiling data tomorrow once I have 
resolved the problem of the testing framework interfering with myfaces 
on production level performancewise.


Werner

Am 10/13/11 11:20 PM, schrieb Werner Punz:
> Yes from what i gather one of the issues they had in the slides was the
> overall page size. The question there is more along the lines what did
> they count, just the rendered code, or also the includes.
>
> I can help to reduce the size on the JSF.js side. We have some code
> which is not directly active for JSF 2.1 and will very likely become
> part of jsf 2.2 or 2.3. it can be used today already by adding config
> params, Also we have some internationalization
> of the internal error messages.
>
> This code could be externalized into an addition js file for people who
> need it. I think we might save around 20Kbytes that way.
>
> I personally did not think that it was necessary due to the fact that
> the js files usually are gzipped while still bigger than mojarra we
> after gzipping the file talk about sizes of 10-30k etc...
>
> In the end externalizing that code would have caused more burden on the
> users than it would have helped. But given that mojarra just implements
> the raw api and nothing else and does not take some corner conditions
> into consideration and has no browser optimizations they are
> significantly smaller in their jsf.js file and if our size is a problem
> we can reduce it.
>
>
>
>
> Werner
>
>
> Am 10/13/11 11:07 PM, schrieb Leonardo Uribe:
>> Hi
>>
>> I believe probably we already did that. The biggest bottleneck we had
>> was that renderers did many calls to map.get(). Mojarra had an
>> optimization in this part, but MyFaces do not until 2.0.9/2.1.3, so I
>> suppose with the latest code we have better numbers.
>>
>> regards,
>>
>> Leonardo Uribe
>>
>> 2011/10/13 Werner Punz<werner.punz@gmail.com>:
>>> I would be interested as well, especially regarding their test setup, we
>>> basically doubled for instance our ajax performance between 2.0.4 and
>>> the
>>> current state of affairs.
>>>
>>> So it might be interesting to see what testsetup they were using.
>>> From a pure memory point of view we of course have a higher load on the
>>> browser because our ajax implementation deals with things mojarra
>>> does not
>>> and also has an oo layer underneath. But I added browser specific
>>> optimisations so on modern browsers we should be slightly faster than
>>> mojarra in raw ajax processing (at least my personal tests resembled
>>> that
>>> when I did the profiling), while mojarra is sligtly ahead on Firefox
>>> 3.5 and
>>> IE6 and 7.
>>>
>>> Just giving the numbers unfortunately does not help to see where their
>>> bottleneck was they discovered.
>>>
>>>
>>>
>>> Werner
>>>
>>>
>>> Am 10/13/11 10:13 PM, schrieb Andy Schwartz:
>>>>
>>>> Gang -
>>>>
>>>> I recently got wind of the following web framework performance talk
>>>> that was presented at JavaOne:
>>>>
>>>>
>>>> https://oracleus.wingateweb.com/published/oracleus2011/sessions/24122/S24122_234496.pdf
>>>>
>>>>
>>>> I did not attend, but based on the slides it looks like the presenters
>>>> did an very thorough/systematic job of evaluating
>>>> performance/scalability for a handful of web frameworks, including
>>>> JSF. (I also have to say that they slides are simply beautiful -
>>>> wow!)
>>>>
>>>> I wanted to call attention to this talk because I am concerned about
>>>> one aspect of the results. Looking at slide #73, it seems that the
>>>> presenters are seeing significant overhead in the MyFaces test runs
>>>> (ie. vs. equivalent runs in Mojarra). I don't have any details other
>>>> than the $ numbers included in the slides, but seems quite possible
>>>> that there is some low-hanging fruit waiting to be picked (or
>>>> optimized).
>>>>
>>>> Is anyone acquainted with the presenters? Perhaps it would be
>>>> worthwhile to contact them to see whether it would be possible to take
>>>> a closer look at the test case?
>>>>
>>>> Andy
>>>>
>>>
>>>
>>>
>>
>
>
>