hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grace <syso...@gmail.com>
Subject Re: Is there any performance issue with Jrockit JVM for Hadoop
Date Sat, 16 May 2009 07:40:43 GMT
To follow up this question, I have also asked help on Jrockit forum. They
kindly offered some useful and detailed suggestions according to the JRA
results. After updating the option list, the performance did become better
to some extend. But it is still not comparable with the Sun JVM. Maybe, it
is due to the use case with short duration and different implementation in
JVM layer between Sun and Jrockit. I would like to be back to use Sun JVM
currently. Thanks all for your time and help.

On Tue, May 12, 2009 at 1:21 PM, Grace <sysolve@gmail.com> wrote:

> Thanks quite a lot for your time and kind advices.  You are so right. From
> the Mission Control, I found that the map process allocated 90% large
> objects and 10% small ones. I will try to set the TLA setting to see if it
> works. Thanks again.
>
>
>
> On Tue, May 12, 2009 at 6:43 AM, Scott Carey <scott@richrelevance.com>wrote:
>
>> Before Java 1.6, Jrockit was almost always faster than Sun, and often by a
>> lot (especially during the 1.4 days).  Now, its much more use-case
>> dependant.  Some apps are faster on one than another, and vice-versa.
>>
>> I have tested many other applications with both in the past (and IBM's VM
>> on
>> AIX, and HP's VM on HP-UX), but not Hadoop.  I suppose it just may be a
>> use
>> case that Sun has done a bit better.
>>
>> The Jrockit settings that remain that may be of use are the TLA settings.
>> You can use Mission Control to do a memory profile and see if the average
>> object sizes are large enough to warrant increasing the thread local
>> object
>> size thresholds.  That's the only major tuning knob I recall that I don't
>> see below.  If Hadoop is creating a lot of medium sized (~1000 byte to
>> 32kbyte) objects Jrockit isn't so optimized by default for that.
>>
>> You should consider sending the data to the Jrockit team.  They are
>> generally on the lookout for example use-cases where they do poorly
>> relative
>> to Sun.  However, now that they are all under the Oracle-Larry-Umbrella it
>> wouldn't shock me if that changes.
>>
>> On 5/7/09 6:34 PM, "Grace" <sysolve@gmail.com> wrote:
>>
>> > Thanks all for your replying.
>> >
>> > I have run several times with different Java options for Map/Reduce
>> > tasks. However there is no much difference.
>> >
>> > Following is the example of my test setting:
>> > Test A: -Xmx1024m -server -XXlazyUnlocking -XlargePages
>> > -XgcPrio:deterministic -XXallocPrefetch -XXallocRedoPrefetch
>> > Test B: -Xmx1024m
>> > Test C: -Xmx1024m -XXaggressive
>> >
>> > Is there any tricky or special setting for Jrockit vm on Hadoop?
>> >
>> > In the Hadoop Quick Start guides, it says that "JavaTM 1.6.x, preferably
>> > from Sun". Is there any concern about the Jrockit performance issue?
>> >
>> > I'd highly appreciate for your time and consideration.
>> >
>> >
>> > On Fri, May 8, 2009 at 7:36 AM, JQ Hadoop <jq.hadoop@gmail.com> wrote:
>> >
>> >> There are a lot of tuning "knobs" for the JRockit JVM when it comes to
>> >> performance; those tuning can make a huge difference. I'm very
>> interested
>> >> if
>> >> there are some tuning tips for Hadoop.
>> >>
>> >> Grace, what are the parameters that you used in your testing?
>> >>
>> >> Thanks,
>> >> JQ
>> >>
>> >> On Thu, May 7, 2009 at 11:35 PM, Steve Loughran <stevel@apache.org>
>> wrote:
>> >>
>> >>> Chris Collins wrote:
>> >>>
>> >>>> a couple of years back we did a lot of experimentation between sun's
>> vm
>> >>>> and jrocket.  We had initially assumed that jrocket was going to
>> scream
>> >>>> since thats what the press were saying.  In short, what we discovered
>> >> was
>> >>>> that certain jdk library usage was a little bit faster with jrocket,
>> but
>> >> for
>> >>>> core vm performance such as synchronization, primitive operations
the
>> >> sun vm
>> >>>> out performed.  We were not taking account of startup time, just
raw
>> >> code
>> >>>> execution.  As I said, this was a couple of years back so things
may
>> of
>> >>>> changed.
>> >>>>
>> >>>> C
>> >>>>
>> >>>
>> >>> I run JRockit as its what some of our key customers use, and we need
>> to
>> >>> test things. One lovely feature is tests time out before the stack
>> runs
>> >> out
>> >>> on a recursive operation; clearly different stack management at work.
>> >>> Another: no PermGenHeapSpace to fiddle with.
>> >>>
>> >>> * I have to turn debug logging of in hadoop test runs, or there are
>> >>> problems.
>> >>>
>> >>> * It uses short pointers (32 bits long) for near memory on a 64 bit
>> JVM.
>> >> So
>> >>> your memory footprint on sub-4GB VM images is better. Java7 promises
>> >> this,
>> >>> and with the merger, who knows what we will see. This is unimportant
>>  on
>> >>> 32-bit boxes
>> >>>
>> >>> * debug single stepping doesnt work. That's ok, I use functional tests
>> >>> instead :)
>> >>>
>> >>> I havent looked at outright performance.
>> >>>
>> >>> /
>> >>>
>> >>
>> >
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message