From dev-return-39575-apmail-harmony-dev-archive=harmony.apache.org@harmony.apache.org Thu Jul 29 07:49:02 2010 Return-Path: Delivered-To: apmail-harmony-dev-archive@www.apache.org Received: (qmail 19317 invoked from network); 29 Jul 2010 07:49:01 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 29 Jul 2010 07:49:01 -0000 Received: (qmail 60685 invoked by uid 500); 29 Jul 2010 07:49:01 -0000 Delivered-To: apmail-harmony-dev-archive@harmony.apache.org Received: (qmail 60519 invoked by uid 500); 29 Jul 2010 07:48:58 -0000 Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@harmony.apache.org Delivered-To: mailing list dev@harmony.apache.org Received: (qmail 60505 invoked by uid 99); 29 Jul 2010 07:48:57 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Jul 2010 07:48:57 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of alexei.fedotov@gmail.com designates 74.125.82.43 as permitted sender) Received: from [74.125.82.43] (HELO mail-ww0-f43.google.com) (74.125.82.43) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Jul 2010 07:48:51 +0000 Received: by wwb31 with SMTP id 31so47488wwb.0 for ; Thu, 29 Jul 2010 00:48:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=HaZcGUr+QbYUhc5HEP3DSSOcAhfz66dPztPc989Z0DE=; b=OcIWEUQaN0SfPav4cQ97EYu3UkVdUCgdbMqeMBuUhf9pb5jXvOduJJPcWA9bnJqCyn EyEJ02i5IOoNLdZSpLvCtJRdde53SCs/o3QCobclQXdDuTZYhgmXqn9vPAWD493nensP UdU8l6vPozA0Yds1LWJw5aJxbqfJMhJxDL+so= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=wbwQsCWYB7ZZE1ncy9db3mkH65ATxFD+YHp12IXZqynCNER2zL4mtTezIkbknA9VSy eGbL8HM5+i9tiWQZ0gWEKIdFdvVSQoqWTPpcMwXPsDmJtiugQHQO1vSeBXFhlrB8nz7K WGbI83XGp7mJ+o7SQ4Ti8cdmc3H7HWPzFKi/U= MIME-Version: 1.0 Received: by 10.227.128.134 with SMTP id k6mr11833002wbs.23.1280389710741; Thu, 29 Jul 2010 00:48:30 -0700 (PDT) Received: by 10.216.165.79 with HTTP; Thu, 29 Jul 2010 00:48:30 -0700 (PDT) In-Reply-To: References: <4C4FE10F.30104@gmail.com> Date: Thu, 29 Jul 2010 11:48:30 +0400 Message-ID: Subject: Re: [hadoop]Status about hadoop unit testing. From: Alexei Fedotov To: dev@harmony.apache.org Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org 1. IMHO, there is some logic in the substitution you've described. Apache Harmony class library is able to work with J9 JVM. When it comes to DRLVM integration, system class library files are substituted wth DRLVM specific implementations. 2. For your reference, DRLVM specific Object and Thread classes can be found here: [1] http://www.google.com/codesearch/p?hl=3Den#-nnolL0D1xY/trunk/drlvm/vm/v= mcore/src/kernel_classes/javasrc/java/lang/Thread.java -- With best regards / =D3 =CE=C1=C9=CC=D5=DE=DB=C9=CD=C9 =D0=CF=D6=C5=CC=C1= =CE=C9=D1=CD=C9, Alexei Fedotov / =E1=CC=C5=CB=D3=C5=CA =E6=C5=C4=CF=D4=CF=D7, http://www.telecom-express.ru/ http://harmony.apache.org/ http://dataved.ru/ http://klsh.ru/ On Thu, Jul 29, 2010 at 10:18 AM, Charles Lee wrote= : > Hi guys, > > I have some few questions about the Unsafe.java. > > There are Unsafes in repos: one is under modules/suncompact [1], another = is > under the drlvm/vm [2]. After build, [2] is in the kernel.jar, [1] is in = the > suncompact.jar. I suppose kernel.jar has the high priority than > suncompact.jar. (suncompact.jar is commented out in the > bootclasspath.properties, which means we just use [2]) What's the differe= nce > between these two files? > > Give a simple diff, we will find that [1] is more updated than [2]. More > investigation on [1], i found [1] depends on some classes which is provid= ed > by the modules/luni-kernal. I supposed classes in the modules/luni-kernal > are stub class, but I am wrong. =9AUpdates in [1] depends on Objects, Thr= eads > and VM class in modules/luni-kernal. And these three classes can not be > found in drlvm. Due to these 3 classes, simply replace [2] with [1] will > cause NoClassDefFoundError. > > Any idea about this? > > On Wed, Jul 28, 2010 at 4:08 PM, Charles Lee wrot= e: > >> >> >> On Wed, Jul 28, 2010 at 3:49 PM, Tim Ellison wrot= e: >> >>> A good summary, thanks Charles. >>> >>> More below... >>> >>> On 28/Jul/2010 06:49, Charles Lee wrote: >>> > Hi guys, >>> > >>> > I am running hadoop unit testing and trying to fix some failures for = a >>> > while. Combining two great projects excites me a lot. Here is my curr= ent >>> > status: >>> > >>> > Env: >>> > Harmony 6 (requires by hadoop) + linux 32 platform >>> > >>> > Using: run-test-core >>> > Whole test cases: =9A757 >>> > Failures: 8 >>> > Errors: 25 >>> > >>> > To get this result, some code need to be changed in hadoop-common [0]= , >>> some >>> > jiras should be commited [1] and some modules I borrowed from RI [2]. >>> > >>> > [0] >>> > a. UserGroupInformation.java should be changed, because hadoop hard >>> coding >>> > the its login module, using com.sun.security things. >>> > b. reduce the failure trying times from 45 to 2. Because junit testca= se >>> will >>> > be timeout if we are trying 45 times. (ri also failed on this) >>> >>> Maybe raise Hadoop JIRAs? >>> >>> > [1] >>> > a. HARMONY-6529 . hadoop is required jdk1.= 6, >>> > patch available. >>> > b. HARMONY-6561 Fixed >>> > c. HARMONY-6569 Fixed >>> > d. HARMONY-6571 JNDI things >>> > e. HARMONY-6575 It may be a drlvm, in my >>> todo >>> > list. >>> > f. =9AHARMONY-6580 , >>> > HARMONY-6605 >>> > =9AFixed >>> > g. HARMONY-6587 Fixed >>> > h. HARMONY-6604 Behavior different betwee= n >>> ri >>> > and harmony >>> >>> I can help with these. >>> >> >> Great :-) >> >> >>> >>> > [2] >>> > a. add tools.jar, which in the harmony jdk/lib, in the >>> > bootclasspath.properties. Ant try to find the com.sun.tools.javac.Mai= n >>> to >>> > determine whether JAVA_HOME is point to a jre or a jdk. This jar is n= ot >>> from >>> > ri, but we may need to change our layout a little bit. >>> >>> I'm struggling a bit to follow you. >>> >>> Both the RI and harmony have got a /lib/tools.jar, and both >>> contain a type called "com.sun.tools.javac.Main" (we added it in Harmon= y >>> for compatibility). >>> >>> So what is the issue here? >>> >>> >> Ant try to find the com.sun.tools.javac.Main, and suppose this class sho= uld >> be on the classpath. >> We do have a com.sun.tools.javac.Main in jdk/lib/tools.jar, but this >> archive is not included in our classpath. >> >> >>> > b. add ecj.jar to the $ANT_HOME/lib or add it in the >>> > bootclasspath.properties. >>> >>> This is to get the Ant task working? >>> >>> >> Yes. >> >> >>> > c. javax.annotation.processing, javax.lang.model, javax.tools, these >>> package >>> > borrow from RI, which is missing in harmony java6. >>> >>> Ok - not sure if there is an ALv2 version around for these. >>> >>> > d. sun.security.krb5, this package borrow from RI. Do we have couterp= art >>> in >>> > harmony? I have not found one. >>> > e. sun.misc.HexDumpEncoder, this borrow from RI. Missing in harmony >>> java6. >>> > f. =9Asun.reflect.ReflectionFactory, this borrow from RI. Missing in >>> harmony >>> > java6. [3] >>> > g. >>> sun/misc/Unsafe.defineClass(Ljava/lang/String;[BIILjava/lang/ClassLoade= r;Ljava/security/ProtectionDomain;)Ljava/lang/Class; >>> > This method is not defined in our Unsafe. [3] >>> >>> No idea what d,e,f are doing, but seems unfortunate that Hadoop is usin= g >>> them, sounds like a conversation to have with the Hadoop project. >>> >>> We can guess what g would do, and include that in our Unsafe (but again >>> would be interested to know why Hadoop is using this rather than the >>> ClassLoader#defineClass API). >>> >> >> Hadoop is not directly using these packages, but hadoop unit testing is >> using mockito --> objenesis, and objenesis is using these packages. >> >> >>> >>> > h. javadoc and doclet package is borrowed from RI. Missing in harmony >>> java6. >>> > Is is one of our GSoC now? >>> >>> No, but I assume they are not needed for normal operations of Hadoop? >>> >> >> Hadoop using these packages to preprocessor its annotations in files. = =9ANeed >> these packages to compile hadoop using harmony. >> >> >>> >>> > [3] >>> > f and g in [2] is raised from a popular project: mockito. Mockito is >>> using >>> > another project: objenesis. Now g is at the top of my todo list. >>> >>> Ah, so these non-API calls are made by a Hadoop dependency? >>> >> >> Yes. >> >> >>> >>> Regards, >>> Tim >>> >>> >> >> >> -- >> Yours sincerely, >> Charles Lee >> >> > > > -- > Yours sincerely, > Charles Lee >