harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charles Lee <littlee1...@gmail.com>
Subject Re: [hadoop]Status about hadoop unit testing.
Date Thu, 29 Jul 2010 06:18:49 GMT
Hi guys,

I have some few questions about the Unsafe.java.

There are Unsafes in repos: one is under modules/suncompact [1], another is
under the drlvm/vm [2]. After build, [2] is in the kernel.jar, [1] is in the
suncompact.jar. I suppose kernel.jar has the high priority than
suncompact.jar. (suncompact.jar is commented out in the
bootclasspath.properties, which means we just use [2]) What's the difference
between these two files?

Give a simple diff, we will find that [1] is more updated than [2]. More
investigation on [1], i found [1] depends on some classes which is provided
by the modules/luni-kernal. I supposed classes in the modules/luni-kernal
are stub class, but I am wrong.  Updates in [1] depends on Objects, Threads
and VM class in modules/luni-kernal. And these three classes can not be
found in drlvm. Due to these 3 classes, simply replace [2] with [1] will
cause NoClassDefFoundError.

Any idea about this?

On Wed, Jul 28, 2010 at 4:08 PM, Charles Lee <littlee1032@gmail.com> wrote:

>
>
> On Wed, Jul 28, 2010 at 3:49 PM, Tim Ellison <t.p.ellison@gmail.com>wrote:
>
>> A good summary, thanks Charles.
>>
>> More below...
>>
>> On 28/Jul/2010 06:49, Charles Lee wrote:
>> > Hi guys,
>> >
>> > I am running hadoop unit testing and trying to fix some failures for a
>> > while. Combining two great projects excites me a lot. Here is my current
>> > status:
>> >
>> > Env:
>> > Harmony 6 (requires by hadoop) + linux 32 platform
>> >
>> > Using: run-test-core
>> > Whole test cases:  757
>> > Failures: 8
>> > Errors: 25
>> >
>> > To get this result, some code need to be changed in hadoop-common [0],
>> some
>> > jiras should be commited [1] and some modules I borrowed from RI [2].
>> >
>> > [0]
>> > a. UserGroupInformation.java should be changed, because hadoop hard
>> coding
>> > the its login module, using com.sun.security things.
>> > b. reduce the failure trying times from 45 to 2. Because junit testcase
>> will
>> > be timeout if we are trying 45 times. (ri also failed on this)
>>
>> Maybe raise Hadoop JIRAs?
>>
>> > [1]
>> > a. HARMONY-6529 </jira/browse/HARMONY-6529>. hadoop is required jdk1.6,
>> > patch available.
>> > b. HARMONY-6561 </jira/browse/HARMONY-6561> Fixed
>> > c. HARMONY-6569 </jira/browse/HARMONY-6569> Fixed
>> > d. HARMONY-6571 </jira/browse/HARMONY-6571> JNDI things
>> > e. HARMONY-6575 </jira/browse/HARMONY-6575> It may be a drlvm, in my
>> todo
>> > list.
>> > f.  HARMONY-6580 </jira/browse/HARMONY-6580>,
>> > HARMONY-6605</jira/browse/HARMONY-6605>
>> >  Fixed
>> > g. HARMONY-6587 </jira/browse/HARMONY-6587> Fixed
>> > h. HARMONY-6604 </jira/browse/HARMONY-6604> Behavior different between
>> ri
>> > and harmony
>>
>> I can help with these.
>>
>
> Great :-)
>
>
>>
>> > [2]
>> > a. add tools.jar, which in the harmony jdk/lib, in the
>> > bootclasspath.properties. Ant try to find the com.sun.tools.javac.Main
>> to
>> > determine whether JAVA_HOME is point to a jre or a jdk. This jar is not
>> from
>> > ri, but we may need to change our layout a little bit.
>>
>> I'm struggling a bit to follow you.
>>
>> Both the RI and harmony have got a <JAVA_HOME>/lib/tools.jar, and both
>> contain a type called "com.sun.tools.javac.Main" (we added it in Harmony
>> for compatibility).
>>
>> So what is the issue here?
>>
>>
> Ant try to find the com.sun.tools.javac.Main, and suppose this class should
> be on the classpath.
> We do have a com.sun.tools.javac.Main in jdk/lib/tools.jar, but this
> archive is not included in our classpath.
>
>
>> > b. add ecj.jar to the $ANT_HOME/lib or add it in the
>> > bootclasspath.properties.
>>
>> This is to get the Ant <javac> task working?
>>
>>
> Yes.
>
>
>> > c. javax.annotation.processing, javax.lang.model, javax.tools, these
>> package
>> > borrow from RI, which is missing in harmony java6.
>>
>> Ok - not sure if there is an ALv2 version around for these.
>>
>> > d. sun.security.krb5, this package borrow from RI. Do we have couterpart
>> in
>> > harmony? I have not found one.
>> > e. sun.misc.HexDumpEncoder, this borrow from RI. Missing in harmony
>> java6.
>> > f.  sun.reflect.ReflectionFactory, this borrow from RI. Missing in
>> harmony
>> > java6. [3]
>> > g.
>> sun/misc/Unsafe.defineClass(Ljava/lang/String;[BIILjava/lang/ClassLoader;Ljava/security/ProtectionDomain;)Ljava/lang/Class;
>> > This method is not defined in our Unsafe. [3]
>>
>> No idea what d,e,f are doing, but seems unfortunate that Hadoop is using
>> them, sounds like a conversation to have with the Hadoop project.
>>
>> We can guess what g would do, and include that in our Unsafe (but again
>> would be interested to know why Hadoop is using this rather than the
>> ClassLoader#defineClass API).
>>
>
> Hadoop is not directly using these packages, but hadoop unit testing is
> using mockito --> objenesis, and objenesis is using these packages.
>
>
>>
>> > h. javadoc and doclet package is borrowed from RI. Missing in harmony
>> java6.
>> > Is is one of our GSoC now?
>>
>> No, but I assume they are not needed for normal operations of Hadoop?
>>
>
> Hadoop using these packages to preprocessor its annotations in files.  Need
> these packages to compile hadoop using harmony.
>
>
>>
>> > [3]
>> > f and g in [2] is raised from a popular project: mockito. Mockito is
>> using
>> > another project: objenesis. Now g is at the top of my todo list.
>>
>> Ah, so these non-API calls are made by a Hadoop dependency?
>>
>
> Yes.
>
>
>>
>> Regards,
>> Tim
>>
>>
>
>
> --
> Yours sincerely,
> Charles Lee
>
>


-- 
Yours sincerely,
Charles Lee

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message