From dev-return-39574-apmail-harmony-dev-archive=harmony.apache.org@harmony.apache.org Thu Jul 29 06:19:26 2010 Return-Path: Delivered-To: apmail-harmony-dev-archive@www.apache.org Received: (qmail 81190 invoked from network); 29 Jul 2010 06:19:26 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 29 Jul 2010 06:19:26 -0000 Received: (qmail 82477 invoked by uid 500); 29 Jul 2010 06:19:26 -0000 Delivered-To: apmail-harmony-dev-archive@harmony.apache.org Received: (qmail 82133 invoked by uid 500); 29 Jul 2010 06:19:23 -0000 Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@harmony.apache.org Delivered-To: mailing list dev@harmony.apache.org Received: (qmail 82124 invoked by uid 99); 29 Jul 2010 06:19:22 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Jul 2010 06:19:22 +0000 X-ASF-Spam-Status: No, hits=4.4 required=10.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of littlee1032@gmail.com designates 74.125.83.49 as permitted sender) Received: from [74.125.83.49] (HELO mail-gw0-f49.google.com) (74.125.83.49) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Jul 2010 06:19:14 +0000 Received: by gwb10 with SMTP id 10so271gwb.36 for ; Wed, 28 Jul 2010 23:18:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=gAaJIcSPak7PsrtwrRov7WfqYNT3tEJnH1XXqeET+o8=; b=fxQHe4RmJzLjiq9qed48455H0nT+REmmnJpTvMQnx7QLiL272uktT0nrC66HlxCu7J V/0S+dilTfsSVGIryvKrn4HaHrbdkjTrhklTJi30q1Hh30d1PfD/uuFiF6yzBcpqC9co vMbHhyNH2lhHHNImMlI4K8eCZ8fZNnRV0YPNg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=nu68jr3dIHUJxOnMEKN3kl89uATgX7Yelw/NwumtN5Y3eT/0bU5ZD1kNU2HgY7pKzE vVfOlji1H8ruGuU+RvOuq/udOISZToWPmp0Hu4GUzFwaMpRWWIIUBXsVYwjhY+KJH6WL NfLNvy467+96Uiy98CyLgRjWYHujDOOrO5c9Q= MIME-Version: 1.0 Received: by 10.90.49.19 with SMTP id w19mr382089agw.25.1280384329740; Wed, 28 Jul 2010 23:18:49 -0700 (PDT) Received: by 10.90.63.1 with HTTP; Wed, 28 Jul 2010 23:18:49 -0700 (PDT) In-Reply-To: References: <4C4FE10F.30104@gmail.com> Date: Thu, 29 Jul 2010 14:18:49 +0800 Message-ID: Subject: Re: [hadoop]Status about hadoop unit testing. From: Charles Lee To: dev@harmony.apache.org Content-Type: multipart/alternative; boundary=00163630f29726ccd1048c80b50a X-Virus-Checked: Checked by ClamAV on apache.org --00163630f29726ccd1048c80b50a Content-Type: text/plain; charset=ISO-8859-1 Hi guys, I have some few questions about the Unsafe.java. There are Unsafes in repos: one is under modules/suncompact [1], another is under the drlvm/vm [2]. After build, [2] is in the kernel.jar, [1] is in the suncompact.jar. I suppose kernel.jar has the high priority than suncompact.jar. (suncompact.jar is commented out in the bootclasspath.properties, which means we just use [2]) What's the difference between these two files? Give a simple diff, we will find that [1] is more updated than [2]. More investigation on [1], i found [1] depends on some classes which is provided by the modules/luni-kernal. I supposed classes in the modules/luni-kernal are stub class, but I am wrong. Updates in [1] depends on Objects, Threads and VM class in modules/luni-kernal. And these three classes can not be found in drlvm. Due to these 3 classes, simply replace [2] with [1] will cause NoClassDefFoundError. Any idea about this? On Wed, Jul 28, 2010 at 4:08 PM, Charles Lee wrote: > > > On Wed, Jul 28, 2010 at 3:49 PM, Tim Ellison wrote: > >> A good summary, thanks Charles. >> >> More below... >> >> On 28/Jul/2010 06:49, Charles Lee wrote: >> > Hi guys, >> > >> > I am running hadoop unit testing and trying to fix some failures for a >> > while. Combining two great projects excites me a lot. Here is my current >> > status: >> > >> > Env: >> > Harmony 6 (requires by hadoop) + linux 32 platform >> > >> > Using: run-test-core >> > Whole test cases: 757 >> > Failures: 8 >> > Errors: 25 >> > >> > To get this result, some code need to be changed in hadoop-common [0], >> some >> > jiras should be commited [1] and some modules I borrowed from RI [2]. >> > >> > [0] >> > a. UserGroupInformation.java should be changed, because hadoop hard >> coding >> > the its login module, using com.sun.security things. >> > b. reduce the failure trying times from 45 to 2. Because junit testcase >> will >> > be timeout if we are trying 45 times. (ri also failed on this) >> >> Maybe raise Hadoop JIRAs? >> >> > [1] >> > a. HARMONY-6529 . hadoop is required jdk1.6, >> > patch available. >> > b. HARMONY-6561 Fixed >> > c. HARMONY-6569 Fixed >> > d. HARMONY-6571 JNDI things >> > e. HARMONY-6575 It may be a drlvm, in my >> todo >> > list. >> > f. HARMONY-6580 , >> > HARMONY-6605 >> > Fixed >> > g. HARMONY-6587 Fixed >> > h. HARMONY-6604 Behavior different between >> ri >> > and harmony >> >> I can help with these. >> > > Great :-) > > >> >> > [2] >> > a. add tools.jar, which in the harmony jdk/lib, in the >> > bootclasspath.properties. Ant try to find the com.sun.tools.javac.Main >> to >> > determine whether JAVA_HOME is point to a jre or a jdk. This jar is not >> from >> > ri, but we may need to change our layout a little bit. >> >> I'm struggling a bit to follow you. >> >> Both the RI and harmony have got a /lib/tools.jar, and both >> contain a type called "com.sun.tools.javac.Main" (we added it in Harmony >> for compatibility). >> >> So what is the issue here? >> >> > Ant try to find the com.sun.tools.javac.Main, and suppose this class should > be on the classpath. > We do have a com.sun.tools.javac.Main in jdk/lib/tools.jar, but this > archive is not included in our classpath. > > >> > b. add ecj.jar to the $ANT_HOME/lib or add it in the >> > bootclasspath.properties. >> >> This is to get the Ant task working? >> >> > Yes. > > >> > c. javax.annotation.processing, javax.lang.model, javax.tools, these >> package >> > borrow from RI, which is missing in harmony java6. >> >> Ok - not sure if there is an ALv2 version around for these. >> >> > d. sun.security.krb5, this package borrow from RI. Do we have couterpart >> in >> > harmony? I have not found one. >> > e. sun.misc.HexDumpEncoder, this borrow from RI. Missing in harmony >> java6. >> > f. sun.reflect.ReflectionFactory, this borrow from RI. Missing in >> harmony >> > java6. [3] >> > g. >> sun/misc/Unsafe.defineClass(Ljava/lang/String;[BIILjava/lang/ClassLoader;Ljava/security/ProtectionDomain;)Ljava/lang/Class; >> > This method is not defined in our Unsafe. [3] >> >> No idea what d,e,f are doing, but seems unfortunate that Hadoop is using >> them, sounds like a conversation to have with the Hadoop project. >> >> We can guess what g would do, and include that in our Unsafe (but again >> would be interested to know why Hadoop is using this rather than the >> ClassLoader#defineClass API). >> > > Hadoop is not directly using these packages, but hadoop unit testing is > using mockito --> objenesis, and objenesis is using these packages. > > >> >> > h. javadoc and doclet package is borrowed from RI. Missing in harmony >> java6. >> > Is is one of our GSoC now? >> >> No, but I assume they are not needed for normal operations of Hadoop? >> > > Hadoop using these packages to preprocessor its annotations in files. Need > these packages to compile hadoop using harmony. > > >> >> > [3] >> > f and g in [2] is raised from a popular project: mockito. Mockito is >> using >> > another project: objenesis. Now g is at the top of my todo list. >> >> Ah, so these non-API calls are made by a Hadoop dependency? >> > > Yes. > > >> >> Regards, >> Tim >> >> > > > -- > Yours sincerely, > Charles Lee > > -- Yours sincerely, Charles Lee --00163630f29726ccd1048c80b50a--