From kato-dev-return-246-apmail-incubator-kato-dev-archive=incubator.apache.org@incubator.apache.org Thu Oct 29 11:32:27 2009 Return-Path: Delivered-To: apmail-incubator-kato-dev-archive@minotaur.apache.org Received: (qmail 15783 invoked from network); 29 Oct 2009 11:32:27 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 29 Oct 2009 11:32:27 -0000 Received: (qmail 219 invoked by uid 500); 29 Oct 2009 11:32:27 -0000 Delivered-To: apmail-incubator-kato-dev-archive@incubator.apache.org Received: (qmail 107 invoked by uid 500); 29 Oct 2009 11:32:27 -0000 Mailing-List: contact kato-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: kato-dev@incubator.apache.org Delivered-To: mailing list kato-dev@incubator.apache.org Received: (qmail 99986 invoked by uid 99); 29 Oct 2009 11:32:27 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Oct 2009 11:32:27 +0000 X-ASF-Spam-Status: No, hits=-4.0 required=10.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [134.134.136.24] (HELO mga09.intel.com) (134.134.136.24) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Oct 2009 11:32:15 +0000 Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP; 29 Oct 2009 04:15:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.44,645,1249282800"; d="scan'208";a="564646215" Received: from irsmsx602.ger.corp.intel.com ([163.33.3.242]) by orsmga001.jf.intel.com with ESMTP; 29 Oct 2009 04:30:51 -0700 Received: from irsmsx504.ger.corp.intel.com ([163.33.3.240]) by irsmsx602.ger.corp.intel.com ([163.33.3.242]) with mapi; Thu, 29 Oct 2009 11:31:52 +0000 From: "Bobrovsky, Konstantin S" To: "kato-spec@incubator.apache.org" , "kato-dev@incubator.apache.org" Date: Thu, 29 Oct 2009 11:31:47 +0000 Subject: RE: TCK and all that Thread-Topic: TCK and all that Thread-Index: AcpTJ0PPDWYRfA/1R2ydClRd0fYS5QFXH7EwAAHffjA= Message-ID: References: <4AE07161.8080302@stoo.me.uk> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org > (2) use RI to read the snapshot and retrieve necessary information, the= n Not 'RI', but 'KATO implementation to be tested', of course. Sorry for conf= usion. Thanks, Konst =20 Intel Novosibirsk Closed Joint Stock Company Intel A/O Registered legal address: Krylatsky Hills Business Park,=20 17 Krylatskaya Str., Bldg 4, Moscow 121614,=20 Russian Federation =20 >-----Original Message----- >From: Bobrovsky, Konstantin S [mailto:konstantin.s.bobrovsky@intel.com] >Sent: Thursday, October 29, 2009 5:28 PM >To: kato-spec@incubator.apache.org; kato-dev@incubator.apache.org >Subject: RE: TCK and all that > >Hi Stuart, > >one of the approaches to write TCK tests for the "1.0" mode I can think of >is making the test application retrieve an log information to be checked >(using standard Java API means) to some "golden" file, then expecting the >TCK test obtain the same (or "similar") information from a "snapshot" usin= g >the RI API. > >In more details: >There are several agents: > - KATO implementation to be tested > - Java Runtime which runs the tests and is coupled with the KATO > implementation > - test application (a set of classes run as a sample payload application) > - TCK test, which can work in 2 modes: > (1) run the test application, retrieve and log all the information > to be checked in mode (2) to a "golden file" (should be stable > between invokations), have the snapshot to be generated by some > means. > (2) use RI to read the snapshot and retrieve necessary information, the= n > Compare what's retrieved with the golden file >Each such test is run 2 times: in mode (1) to generate the golden file and >the dump, and in mode (2) when actual API implementation verification >happens. > >Of course, not entire API can be tested this way, but a fair part can. The >main question is what kind of information can be obtained in mode (1), >which can later be retrieved from the snapshot. What comes to mind is: > - all alive threads running in the test application and their Java stack= s > using java.lang.Thread.getAllStackTraces. For threads with pre-defined > names which the test recognizes as "special" ones (and which are > created by the test application), some field values or other details > can be queried > - a subset of live objects created by the test application > - ... many more if JVMTI or JMX is used as the information provider > in mode (1) > >Thanks, >Konst > >Intel Novosibirsk >Closed Joint Stock Company Intel A/O >Registered legal address: Krylatsky Hills Business Park, >17 Krylatskaya Str., Bldg 4, Moscow 121614, >Russian Federation > > >>-----Original Message----- >>From: Stuart Monteith [mailto:stukato@stoo.me.uk] >>Sent: Thursday, October 22, 2009 9:51 PM >>To: kato-dev@incubator.apache.org; kato-spec@incubator.apache.org >>Subject: TCK and all that >> >>Hi everybody, >> >> Apologies for the cross posting - but the implementation of the TCK >>is relevant to both camps. >>One of the things we need to do before making a release available is to >>produce a TCK. By "TCK" I mean >>a test harness that will: >> >>1. Setup a JVM into a known configuration and generate a dump. >>2. Open a dump with the Kato API and run unit tests against it. >> >>Whether or not these are just functional tests or technology compliance >>is something I will just gloss over for now >>as just a classification issue. >> >>As we all know, there is a problem with optionality in the API. There is >>information that will be lost when using >>certain dump types. >> >>Steve has suggested two modes for running the TCK. >> >> Legacy >>=3D=3D=3D=3D=3D=3D=3D=3D >> >>This mode runs through the API and confirms that the basic API behaves >>as it is intended to do so. >>It checks that the rules of the API are adhered to, but doesn't expect >>any method will return any >>particular piece of information. >> >>For example: >> List ImageAddressSpace.getProcesses(); >> >>processes would be checked to ensure that it is not null. If there were >>contents >>they would be queried, and their methods executed. >> >>For example: >> String ImageProcess.getID(); >> >>Would be expected to return a String or throw a DataUnavailable or >>CorruptDataException exception. >>The actual contents of the string wouldn't be checked - just so long as >>it isn't null. >>(incidentally, actually checking the value in this example for >>correctness is difficult). >> >> 1.0 >>=3D=3D=3D=3D=3D >> >>This is a test for exact compliance with the API. All of the information >>that can be retrieved from the dump >>must be retrieved and it must all be correct. >> >>With some caveats. >> >>Like with ImageProcess.getID(), there is some information that is poorly >>specified or difficult to know in >>advance. >> >>The poorly specified aspects of the API are: >> Platform specific items. >> For example: >> >> Implementation specific items: >> toString() methods - what do they print? >> >>Things that are difficult to know in advance: >> Native stack frames - ordering and naming. >> JavaThreads - there will be threads that are implementation specific. >> >>There problems aren't intractable. A certain amount of flexibility will >>be necessary to accommodate variations. >>We should expect there to be more than what we expect, so we should just >>look for what we expect to be there, >>and ignore the rest. >> >>The poorly specified aspects of the API should just be more precisely >>specified. Although it means that the TCK will >>have to be aware of the different platforms and their behaviour. >> >> >>I'm not a great fan of the "Legacy" mode. While it may have a basic use >for >>testing basic behaviour, I would like for there to be stronger statement >>about what we expect the API to do. >> >>For example, if we call JavaRuntime.getThreads(), I would expect a >>List to be returned. However, if it wasn't empty, I would wan= t >>the threads we put into the testcase to be there and identifiable by thei= r >>(correct) names. >> >>i.e. if there is something to test we should test it. >> >>I call this mode "Permissive" (alternatives: lenient, acquiescent, lax, >>liberal, tolerant) >> >>There are, of course, problems. But I would expect that is this mode >wasn't >>possible or advisable, it would be improbable that a program could be >>writtent to call the API. >> >>The end result is that if you get information out of the API, it must be >>correct. Where you don't, it is ignored. >> >>Thoughts? >> >> >>Thoughts? >> >>Regards, >> Stuart >> >> >> >>-- >>Stuart Monteith >>http://blog.stoo.me.uk/