Return-Path: Delivered-To: apmail-harmony-commits-archive@www.apache.org Received: (qmail 94248 invoked from network); 17 Aug 2007 09:57:22 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 17 Aug 2007 09:57:22 -0000 Received: (qmail 140 invoked by uid 500); 17 Aug 2007 09:57:19 -0000 Delivered-To: apmail-harmony-commits-archive@harmony.apache.org Received: (qmail 128 invoked by uid 500); 17 Aug 2007 09:57:19 -0000 Mailing-List: contact commits-help@harmony.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@harmony.apache.org Delivered-To: mailing list commits@harmony.apache.org Received: (qmail 119 invoked by uid 99); 17 Aug 2007 09:57:19 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Aug 2007 02:57:19 -0700 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO eris.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Aug 2007 09:57:19 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 76CB61A9827; Fri, 17 Aug 2007 02:56:37 -0700 (PDT) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r566994 [6/8] - in /harmony/standard/site: docs/ docs/documentation/ docs/documentation/milestones/ docs/subcomponents/buildtest/ docs/subcomponents/classlibrary/ docs/subcomponents/drlvm/ docs/subcomponents/jchevm/ docs/subcomponents/stres... Date: Fri, 17 Aug 2007 09:56:23 -0000 To: commits@harmony.apache.org From: nadinem@apache.org X-Mailer: svnmailer-1.1.0 Message-Id: <20070817095637.76CB61A9827@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Modified: harmony/standard/site/xdocs/subcomponents/drlvm/Jitrino_PMF.html URL: http://svn.apache.org/viewvc/harmony/standard/site/xdocs/subcomponents/drlvm/Jitrino_PMF.html?view=diff&rev=566994&r1=566993&r2=566994 ============================================================================== --- harmony/standard/site/xdocs/subcomponents/drlvm/Jitrino_PMF.html (original) +++ harmony/standard/site/xdocs/subcomponents/drlvm/Jitrino_PMF.html Fri Aug 17 02:56:16 2007 @@ -1,1306 +1,1306 @@ - - - - - - - - - Jitrino Pipeline Management Framework - - - -

Jitrino Pipeline Management Framework

- -
    -
  1. Revision History
    2. -
  2. About This Document -
      -
    1. Purpose
      2. -
    2. Intended Audience
      3. -
    3. Documentation Conventions
      4. -
    -
    3. -
  3. Overview -
      -
    1. Key Elements
      2. -
    2. Implementation Notes
      3. -
    3. Pipeline Life-cycle -
        -
      1. Initialization
        2. -
      2. Method Compilation
        3. -
      3. De-initialization
        4. -
      -
      4. -
    -
    4. -
  4. Command-line Interface
    5. -
  5. PMF Commands -
      -
    1. Path Definition
      2. -
    2. Filter Definition
      3. -
    3. Argument Definition
      4. -
    -
    6. -
  6. Logging System -
      -
    1. Defining Streams
      2. -
    2. Using Filters in Logging -
    3. Assigning a Stream to a File -
        -
      1. Using Filename Masks
        2. -
      2. Filename Review
        3. -
      -
      4. -
    4. Using Streams
      5. -
    -
    7. -
  7. Appendix -
      -
    1. - General Command-line Syntax Specification
      2. -
    -
    8. -
-

Revision History

- - - - - - - - - - - - - - - - - -
VersionVersion InformationDate
Initial versionSergey Ivashin, Svetlana - Konovalova: document created.October 30 , 2006
- -

About This Document

- -

Purpose

- -

The document is a comprehensive description of the - pipeline management framework that provides complete - control over just-in-time compilation process through the - Java* property mechanism. The description - covers the PMF application to the command-line interface and to - the Jitrino logging system.

- -

PMF is introduced in the Jitrino component description, - whereas the current document gives details on its internal - design and specifics. The current document adds examples and - usage instructions to the general definition of PMF given in - the component description.

- -

Intended Audience

- -

The target audience for the document includes a wide - community of DRLVM developers interested in understanding - Jitrino internals and in improving them. The document assumes - that readers understand the concepts of - PMF and just-in-time - compilation.

- -

Documentation - Conventions

- -

This document uses the unified - conventions for the DRL documentation kit.

- -

Back to Top

- -

Overview

- -

The document contains the detailed description of the - Jitrino pipeline management framework (PMF). For a high-level - PMF overview, refer to the Jitrino - document.

- -

Key Elements

- -

This section contains a detailed description of the PMF key - elements.

- -
-
Pipeline
- -
-

The pipeline is a sequence of steps - representing the compilation process for a set of - methods. You can create several pipelines for a single - JIT, targeting each to compile a particular set of - methods. Each instance of the JIT compiler has one - common pipeline, which has an empty method - filter. Thus, the common pipeline is applied if a - compiled method matched no method filter of other - pipelines. For more information on the common pipeline, - refer to the - Initialization section of - the current document.

-
- -
Method Filter
- -
-

The method filter is used to choose a - particular pipeline for a method being compiled. Filter - elements make up a filter expression, as follows:
- <class_name><method_name>< - method_signature>.
- Any or all of these elements can be blank. If all - elements of a filter expression are empty strings, the - filter is empty. Empty filters are used in the - compilation of a common pipeline.

-
- -
Pipeline Step
- -
-

The pipeline step is a pipeline element that - stores a reference to an action object and its full - name. In the pipeline definition a step is represented - by its textual name.

-
- -
Action
- -
-

The action is an object representing a - certain independent transformation of compiled code. - Each action must be declared as a subclass of the - Jitrino::Action class. Therefore, all - actions have a common set of properties and methods, - namely:

- -
    -
  • The name property used to assign a - unique name to each JIT action
    • - -
  • The init() and - deinit() methods used to - initialize and - de-initialize the - JIT
    • - -
  • The createSession() method used to - create - SessionAction
    • -
- -

Note

- -

Actions do not have the - run() method and cannot be called directly - in the pipeline. The SessionAction object - is used instead.

- -

The system creates Action objects once - per JIT execution and uses them to store data between - compilation sessions.

- -

SessionAction

- -

A SessionAction object created by PMF - to transform code in a particular compilation session. - SessionAction must be declared as a - subclass of the Jitrino::SessionAction - class, which has the following methods:

- -
    -
  • The getAction() method to get the - reference to the parent Action - object
    • - -
  • The run() method to do the work - for the action object
    • -
- -

Instances of SessionAction are created - at the start of method compilation and destroyed at the - end of compilation. In other words, they are created - and destroyed every time when a pipeline with the - corresponding action is selected for method - compilation. Therefore, no data can be saved in this - object between compilation sessions.

-
- -
IAction
- -
-

The IAction class is a common ancestor - of the Action and - SessionAction classes, which contains - service methods that may be accessed from both child - classes; for example, argument processing methods or - methods for obtaining a reference to the corresponding - pipeline.

-
-
- -

package

- -

Figure 1: The Role of Pipelines in the JIT - Compilation Process

- -

Figure 1 demonstrates the role of pipelines in the JIT - compilation process.

-

Back to Top

- -

Implementation - Notes

- -

When using the PMF implementation, please note the - following:

- -
    -
  • The common pipeline is mandatory, others are - optional.
    • - -
  • An optional pipeline must be assigned to a unique and - non-empty filter expression.
    • - -
  • Different pipeline steps can refer to the same - action.
    • - -
  • Different pipelines can have different sets of steps - and different orders of steps.
    • - -
  • Each pipeline step can be supplied with a unique - parameter.
    • -
- -

Back to Top

- -

Pipeline Life-cycle

- -

Initialization

- -

The common pipeline is created at the JIT initialization. - All other pipelines are created just before their first use. - Therefore, an optional pipeline is only initialized when a - compiled method matches its filter.

- -

During pipeline creation, the compiler iterates over all its - steps. For each step, the compiler creates an instance of the - referenced Action object and calls the - Action::init() method. At this point, all action - parameters are available and the Action::init() - method can use any of the IAction::getArg methods - to retrieve action parameters. For efficiency purposes, - parameters should be processed at the initialization phase, not - during compilation of each method.

- -

Note

- -

You can call the - IAction::getArg method from - SessionAction::run() during compilation of a - method, but it is a less effective way of parameter processing. - The SessionAction object has access to its parent - Action object and therefore, can access data - prepared during the initialization phase.

- -

Method Compilation

- -

At this stage, the compiler selects the pipeline for the - given method using the filter expression in the method filter - and proceeds to compile a method. If the method matches several - filters, then the pipeline with the longest filter expression - is selected, that is PMF always chooses the filter with more - characters in its expression. PMF prohibits creation of - different pipelines with the same filter expression.

- -

To compile the method, JIT executes the selected pipeline: - iterates over all its steps, creates a - SessionAction object for each step, calls the - run() method in each object, and destroys it.

- -

De-initialization

- -

At the JIT de-initialization phase all created pipelines are - destroyed. The Action::deinit() method is called - for every step in a pipeline during its de-initialization.

- -

Back to Top

- -

Command-line - Interface

- -

The command-line interface is based on system - properties defined by "property = value" relations. At the - initialization phase, all properties are stored in the table of - VM properties. Depending on the configuration mode, VM can - create several JITs. In this case, each JIT reads this table - and selects properties independently; therefore, different JITs - have different property sets.

- -

The general form of the JIT command is the following:

- 
--XX:jit.<JIT>.<pipeline>.<parameter>=<value>
-
- -

Where

- -
    -
  • - -XX:jit. is the mandatory prefix enabling - JIT commands selection from VM commands. - -

    Note

    - -

    The Jitrino command-line interface - ignores all commands that do not start with this - prefix.

    -
    • - -
  • - <JIT> specifies the JIT name. - - -

    Note

    - -

    JITs are created by - Execution Manager (EM) using - information from the command line and/or - - execution manager configuration - file.

    -
    • - -
  • <pipeline> specifies the - pipeline name.
    • - -
  • <parameter> and - <value> are used to construct one - of the three available basic commands: - filter definition, - path definition and - argument - definition.
    • -
- -

Both <JIT> and - <pipeline> elements define the - command scope in the following way:

- -
    -
  • If the <JIT> element is - omitted, the scope of the command consists of all existing - JITs.
    • - -
  • If the <pipeline> element is - omitted, the scope of the command consists of all pipelines - existing in the specified JIT.
    • - -
  • If both <JIT> and - <pipeline> are omitted, the scope - of the command consists of all pipelines of all JITs.
    -
    • -
- -

Commands can be specified on the command line or in the EM - configuration file. Most important commands are usually placed - in the - - EM configuration file, so Jitrino can be used - without any commands on the command line.

- -

The order of commands on the command line and in the - external file is not significant.

- -

Another way to deal with a set of commands is to store it in - a command file and read this file with a special read - command:

- 
--XX:jit.read=<command filename>
-
- -

Note

- -

Write commands in the command file without - the prefix -XX:jit. The read command - itself can be used in the command file. The command file can - contain empty lines and comment lines starting with - # or //.

- -

For information on the general command line syntax - specification, refer to the - Appendix.

- -

Back to Top

- -

PMF Commands

- -

To control the JIT compilation process, PMF uses - path definition, - filter definition and - argument definition - commands.

- -

Path Definition

- -

A pipeline can be described as an ordered set of terminal - leafs of a path tree. The tree is a purely abstract concept - introduced to describe the pipeline construction process. Each - non-terminal node of the tree is called a path and - defined by the following path definition command:

- 
--XX:jit.<JIT>.<pipeline>.path.<name>=<child1>,<child2>,...
-
- -

The given command introduces a node with the specified name - and enumerates all its direct descendants - children. If a - child is a terminal node, it is treated as the name of the - action; if not, then the child must be another path.

- -

For each pipeline, define one path with the - <name> element omitted, which is called the - root path. Each path must have a unique definition.

- -

Each action in the pipeline has a unique full name - consisting of all node names from the root to the action - itself. The root node is excluded because it has no name. More - than one instance of the same action can exist, as long as each - instance has a unique full name.

- -

Example

- -

OPT is the JIT name - and a, b and - c are the action names.

- 
--XX:jit.OPT.path=M,N
--XX:jit.OPT.path.M=a
--XX:jit.OPT.path.N=b,c
-
- -

The given commands describe the simple - path tree:

- -

package

- -

Figure 2: A Path Tree of a Common - Pipeline

- -

The following resulting pipeline is - constructed from the aforementioned tree:
- a, b, - c

- -

The pipeline above is common, because it - has no method filter. Each action of the given pipeline has a - unique full name.

- - - - - - - - - - - - - - - - - - - - - - - - - -
ActionFull Name
aM.a
bN.b
cN.c
- -

Back to Top

- -

Filter Definition

- -

To define a filter, use the following command format:

- 
--XX:jit.<JIT>.<pipeline>.filter=<class>.<method><signature>
-
- -

Where

- -
    -
  • <pipeline> is the required - element, which cannot be omitted.
    • - -
  • <class> is the name of the - class constituting the filter expression.
    • - -
  • <method> is the name of the - method constituting the filter expression.
    • - -
  • <signature> is the signature - of the method constituting the filter expression.
    • -
- -

Note

- -

In the filter definition, the - method and the signature elements - have no period . between them.

- -

You must follow the VM notation for the described elements. - You are free to omit some elements, but at least one must be - present.

- -

If each of three elements of the method name starts with the - corresponding element of the filter expression, the compiled - method matches the filter.

- -

Example

- 
--XX:jit.OPT.p.filter=java/lang/Thread.check
-
- -

The following methods can match the - given filter:
- java/lang/ThreadGroup.checkAccess()V
- java/lang/Thread.checkGCWatermark()V
- java/lang/Thread.checkAccess()V

- -

To continue the example, methods can equally match a less - detailed filter expression, such as - java/lang.check. However, PMF would use the more - specific filter, as described in the - Method Compilation - section.

- -

Example

- 
--XX:jit.OPT.path=M,N
--XX:jit.OPT.path.M=a
--XX:jit.OPT.path.N=b,c
--XX:jit.OPT.p.path=M,N,O
--XX:jit.OPT.p.path.O=a
--XX:jit.OPT.p.filter=java/lang/Thread.check
-
- -

The commands above are associated with - the following path trees:

- -

package

- -

Figure 3: An Example of Path Trees for - Multiple Pipelines

- -

The following resulting pipelines are - constructed from the aforementioned trees:
- (common) a, - b, c
- (p)           - a, b, - c, a

- -

Note

- -

Path definitions M and - N in the pipeline p are - inherited from the common pipeline, because they were defined - in commands with the pipeline element omitted, but the root - path definition in the p pipeline was - overridden.

- -

Back to Top

- -

Argument Definition

- -

To control the actions behavior, PMF supports an action - parameters passing convention. The parameter definition command - pattern is the following:

- 
--XX:jit.<JIT>.<pipeline>.arg.<full name>.<param>=<value>
-
- -

Where

- -
    -
  • <param> and <value> are - mandatory elements.
    • - -
  • <JIT>, <pipeline> and - <full name> are optional elements defining the - scope of the both.
    • -
- -

The <full name> element defines the parameter - scope in the following way:

- -
    -
  • If <full name> addresses a certain action, - that is a terminal node, the parameter affects all - instances of this action.
    • - -
  • If <full name> addresses not an action, - but a node with children, the parameter affects this node - and all its descendants.
    • - -
  • If <full name> is omitted, the parameter - affects all actions in the pipeline.
    • -
- -

Example

- 
--XX:jit.OPT.p.arg.a.verify=1
-
- -

The given command sets the - <verify> parameter for both a - instances, M.a and O.a, - to 1. See Figure 3.

- -

The table below gives several examples of essential commands - for the path trees based on Figure 3.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
CommandOmitted ElementDefinition
- -XX:jit.OPT.p.arg.N.b.verify=2 -
- — -
- 		The parameter verify - with value 2 is available only to action - b with the full name - N.b in pipeline - p of JIT - OPT.
- -XX:jit.OPT.p.arg.N.verify=2 -
- — -
- 		The parameter verify - is available for actions b and - c simultaneously.
- -XX:jit.OPT.p.arg.verify=2<full - name>The parameter verify - is available for all actions a, - b, c and - a in pipeline - p.
- -XX:jit.OPT.arg.N.verify=2 - <pipeline>The parameter verify - is available for b and - c actions for common and - p pipelines of JIT - OPT .
- -

The special form of argument definition commands controls - the pipeline construction process:

- 
--XX:jit.<JIT>.<pipeline>.arg.<full name>=on
--XX:jit.<JIT>.<pipeline>.arg.<full name>=off
-
- -

Note

- -

In the given form of the argument - definition, the <param> element is absent and only - two values for <value> exist: on and - off.

- -

The off value disables the addressed node and - all its descendants, while the on value enables - the node and its descendants. When constructed, a pipeline does - not include any disabled actions.

- -

Example

- -

To exclude action b - from the pipeline, apply the following command:

- 
--XX:jit.OPT.p.arg.N.b=off
-
- -

To enable/disable nodes directly in the path definition - command, add state characters immediately after the node name, - videlicet:

- - - - - - - - - - - - - - - - - - - - - - - - - -
CharacterCorresponding Node - Meaning
-
- + -
- 		enabled
-
- — -
- 		disabled
-
- ! -
- 		cannot be disabled
- -

The following command states that c1 is - enabled, c2 is disabled and c3 is - enabled and mandatory:

- 
--XX:jit.<JIT>.<pipeline>.path.<name>=c1+,c2-,c3!
-
- -

If you try to disable any mandatory node with the - off command, the fatal error occurs.

- -

Back to Top

- -

Logging System

- -

The logging system supports diagnostic messages from - pipeline actions. The VM property mechanism fully controls the - system.

- -

Defining Streams

- -

A set of log streams is used for logging in Jitrino. - Each log stream has a name. The name of the log stream is not - related to the name of the file, to which the stream is - assigned. The class Jitrino::LogStream is the - program representation of a log stream. This class is similar - to the std::ostream class in the defining operator - << and other useful methods, such as - printf() and flush(). To obtain the - reference to the underlying std::ostream instance, - call the out() method.

- -

Streams can be accessed by name or by ID. The stream ID can - be obtained from its name by using the - IAction::getLogStreamID() method. For performance - reasons, several most frequently used streams, so-called - known streams, have predefined IDs:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Stream NameStream IDOutput
infoLogStream::INFOThe protocol of compilation: JIT - and pipeline names, the method name and number, and so - on.
rtLogStream::RTRun-time output not related to - compiled methods.
ctLogStream::CTCompile-time diagnostic.
irdump - LogStream::IRDUMPThe dump of internal Jitrino - structures for a compiled method.
dotdump - LogStream::DOTDUMPThe dump of internal Jitrino - structures in the .dot format.
dbgLogStream::DBGDebug information.
- -

To enable outputting log information to a stream, enter a - command of the following syntax:

- 
--XX:jit.<JIT>.<pipeline>.arg.<full name>.log=<stream1>,<stream2>,...
-
- -

The left part of the command specifies the name of the JIT - compiler and the compilation pipeline to be used, whereas the - right part contains the names of the streams that you want to - enable. Use the same syntax to enable a custom stream: write an - arbitrary name for it and enable the stream with this - command.

- -

Note

- -

If program output is directed to a disabled - stream, the output is lost and no error diagnostic is - produced.

- -

The enabling command follows the general scope of rules: the - stream is enabled for the node specified in the command and all - its descendants.

- -

Streams defined for different nodes are different streams, - though they can have the same name.

- -

Example

- -

The following commands enable two - different streams, though both can be accessed from actions - N.b and N.c by the same - name. The streams can be assigned to different files.

- 
--XX:jit.OPT.p.arg.N.b.log=ct
--XX:jit.OPT.p.arg.N.c.log=ct
-
- -

Back to Top

- -

Using Filters in Logging

- -

A method filter can be used to filter - logging by package, class or method name. For logging purposes, define an empty - pipeline with an appropriate method filter, with this pipeline - considered a filter name for simplicity.

- -

Example

-

The following commands implicitly define - an empty pipeline f for filtering purposes, - defines a method filter for this pipeline and enables logging for - the methods corresponding to f.

- - 
-XX:jit.OPT.f.filter=main
--XX:jit.f.arg.log=ct
-
- -

Back to Top

- -

Assigning a - Stream to a File

- -

To assign a stream to a file, use the following command - syntax:

- 
--XX:jit.<JIT>.<pipeline>.arg.<full name>.log.<stream>.file=<filename mask>
-
- -

In the given command, the <filename mask> - element constructs a filename for the stream. All characters - from this mask are copied to the resulting filename literally - except macros, which are expanded and the result of expansion - is copied to the filename. Below is the list of available - macros.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
MacroExpands To
%jit%The name of the JIT that is - compiling the method.
%class%The method class name with - / changed to _
%class_tree%The method class name with - / preserved.
%method%The method name and signature in - VM notation.
%seqnb%The global sequential number of - the current compilation.
%log%The name of the stream.
%thread%The number of the thread - compiling the method.
- -

In most cases, a program can access the stream file from - concurrent threads, so that all stream operations are - synchronized. However, if a filename mask for the stream - contains the %thread% macro, this file is - thread-specific and access to it is not synchronized.

- -

Using Filename - Masks

- -

For all streams, the following default filename mask is - used:

- 
-log/%jit%/%class%/%method%/%log%.log
-
- -

For example, for an enabled stream abc with no - assigned filename mask, the log file is in the same directory - with other known stream files, and its name is - abc.log.

- -

All known streams have hard-coded default filename masks, - described in the table below. These masks are used if no - assignment is made with the corresponding command.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Stream NameDefault Filename Mask
infolog/info.log
rtlog/rt.log
ct - log/%jit%/%class%/%method%/ct.log
irdump - log/%jit%/%class%/%method%/irdump.log
dotdump - log/%jit%/%class%/%method%/.dot [special - case]
- -

Exception

- -

Jitrino writes dot-dumps to the - dotdump stream and produces two .dot - files for each action: before and after execution - of an action. To handle these correctly, the enabled dot-dumper - opens the dotdump stream before executing the - action, writes to the stream and closes it. After executing the - action, the dot-dumper performs the same operation. Each time, - the dot-dumper adds a suffix that indicates the sequence of - actions to the tail of the stream filename .dot, - as follows:

- 
-  .<action_nb>.<action_name>.before.dot
-  .<action_nb>.<action_name>.after.dot
-
- -

In the command above, the <action name> element - is the name of the action and <action_nb> is the - sequential number of the action in the current pipeline.

- -

Filename Review

- -

Names of all open stream files are reviewed at the beginning - of method compilation, because the stream file name can depend - on the name of the compiled method. If the name is changed as - the result of the dependency on the compiled method name, the - file is closed and another file with the new calculated name is - opened. If the name is not changed, no action is performed. If - the same file name is assigned to different streams, a single - file is opened and all the assigned streams use it.

- -

If a file used for logging gets closed as the result of - filename recalculation, its name is saved in a special list - that PMF maintains. For every file opened for logging output, - the PMF checks the list. If the name is found, the file is - opened in the append mode. This approach prevents from - overwriting logging files during a single Jitrino run.

- -

You can use the following commands to control the - append/truncate mode for the log files that are opened for the - first time:

- 
--XX:jit.<JIT>.<pipeline>.arg.<full name>.log.<stream>.append=true
--XX:jit.<JIT>.<pipeline>.arg.<full name>.log.<stream>.append=false
-
- -

Back to Top

- -

Using Streams

- -

A typical usage pattern for Jitrino action is the - following:

- -
    -
  1. Find the required stream using its name or ID.
    2. - -
  2. (optional) Test whether the stream is enabled.
    3. - -
  3. Output to the stream using the insertion operator - << or printf() method.
    4. -
- -

Example

- 
-    LogStream& irdump = log(LogStream::IRDUMP);
-
-    if (irdump.isEnabled()) {
-        irdump << "Opt:   Running " << getName() << stl::endl;
-    }
-
- -

Back to Top

- -

Appendix

- -

- General Command-line Syntax Specification

- 
-<cmd>                     :     <prefix> <filter def>
-                                <prefix> <path def>
-                                <prefix> <arg def>
-                                <prefix> <file read>
-
-<prefix>                  :     -XX:jit.
-
-<base>                    :     [ <JIT> . ] [ <pipeline> . ]
-
-<JIT>                     :     <name>
-
-<pipeline>                :     <name>
-
-
-<filter def>              :     <base> filter = <filter spec>
-
-<filter spec>             :     [ <class> ]  [ .  <method> [ <signature> ] ]
-                                [ <class> ]  [ :: <method> [ <signature> ] ]
-
-
-<path def>                :     <base> path . <node> = <path items>
-                                <base> path = <path items>
-
-<path items>              :     <path item>
-                                <path items> , <path item>
-
-<path item>               :     {<node> | <action >} [{ ! | + | - }]
-
-<node>                    :     <name>
-
-<action>                  :     <name>
-
-
-<arg def>                 :     <base> arg  [ <full name> . ] <arg name> = <arg value>
-                                <base> arg  <full name> = {off | on}
-
-<full name>               :     <path item>
-                                <full name> . <path item>
-
-<arg name>                :     <name>
-
-<arg value>               :     <string>
-
-
-<file read>               :     read = <file name>
-
-
- -

Back to Top

- -

* Other brands and names are the property of - their respective owners.

- - + + + + + + + + + Jitrino Pipeline Management Framework + + + +

Jitrino Pipeline Management Framework

+ +
    +
  1. Revision History
    2. +
  2. About This Document +
      +
    1. Purpose
      2. +
    2. Intended Audience
      3. +
    3. Documentation Conventions
      4. +
    +
    3. +
  3. Overview +
      +
    1. Key Elements
      2. +
    2. Implementation Notes
      3. +
    3. Pipeline Life-cycle +
        +
      1. Initialization
        2. +
      2. Method Compilation
        3. +
      3. De-initialization
        4. +
      +
      4. +
    +
    4. +
  4. Command-line Interface
    5. +
  5. PMF Commands +
      +
    1. Path Definition
      2. +
    2. Filter Definition
      3. +
    3. Argument Definition
      4. +
    +
    6. +
  6. Logging System +
      +
    1. Defining Streams
      2. +
    2. Using Filters in Logging +
    3. Assigning a Stream to a File +
        +
      1. Using Filename Masks
        2. +
      2. Filename Review
        3. +
      +
      4. +
    4. Using Streams
      5. +
    +
    7. +
  7. Appendix +
      +
    1. + General Command-line Syntax Specification
      2. +
    +
    8. +
+

Revision History

+ + + + + + + + + + + + + + + + + +
VersionVersion InformationDate
Initial versionSergey Ivashin, Svetlana + Konovalova: document created.October 30 , 2006
+ +

About This Document

+ +

Purpose

+ +

The document is a comprehensive description of the + pipeline management framework that provides complete + control over just-in-time compilation process through the + Java* property mechanism. The description + covers the PMF application to the command-line interface and to + the Jitrino logging system.

+ +

PMF is introduced in the Jitrino component description, + whereas the current document gives details on its internal + design and specifics. The current document adds examples and + usage instructions to the general definition of PMF given in + the component description.

+ +

Intended Audience

+ +

The target audience for the document includes a wide + community of DRLVM developers interested in understanding + Jitrino internals and in improving them. The document assumes + that readers understand the concepts of + PMF and just-in-time + compilation.

+ +

Documentation + Conventions

+ +

This document uses the unified + conventions for the DRL documentation kit.

+ +

Back to Top

+ +

Overview

+ +

The document contains the detailed description of the + Jitrino pipeline management framework (PMF). For a high-level + PMF overview, refer to the Jitrino + document.

+ +

Key Elements

+ +

This section contains a detailed description of the PMF key + elements.

+ +
+
Pipeline
+ +
+

The pipeline is a sequence of steps + representing the compilation process for a set of + methods. You can create several pipelines for a single + JIT, targeting each to compile a particular set of + methods. Each instance of the JIT compiler has one + common pipeline, which has an empty method + filter. Thus, the common pipeline is applied if a + compiled method matched no method filter of other + pipelines. For more information on the common pipeline, + refer to the + Initialization section of + the current document.

+
+ +
Method Filter
+ +
+

The method filter is used to choose a + particular pipeline for a method being compiled. Filter + elements make up a filter expression, as follows:
+ <class_name><method_name>< + method_signature>.
+ Any or all of these elements can be blank. If all + elements of a filter expression are empty strings, the + filter is empty. Empty filters are used in the + compilation of a common pipeline.

+
+ +
Pipeline Step
+ +
+

The pipeline step is a pipeline element that + stores a reference to an action object and its full + name. In the pipeline definition a step is represented + by its textual name.

+
+ +
Action
+ +
+

The action is an object representing a + certain independent transformation of compiled code. + Each action must be declared as a subclass of the + Jitrino::Action class. Therefore, all + actions have a common set of properties and methods, + namely:

+ +
    +
  • The name property used to assign a + unique name to each JIT action
    • + +
  • The init() and + deinit() methods used to + initialize and + de-initialize the + JIT
    • + +
  • The createSession() method used to + create + SessionAction
    • +
+ +

Note

+ +

Actions do not have the + run() method and cannot be called directly + in the pipeline. The SessionAction object + is used instead.

+ +

The system creates Action objects once + per JIT execution and uses them to store data between + compilation sessions.

+ +

SessionAction

+ +

A SessionAction object created by PMF + to transform code in a particular compilation session. + SessionAction must be declared as a + subclass of the Jitrino::SessionAction + class, which has the following methods:

+ +
    +
  • The getAction() method to get the + reference to the parent Action + object
    • + +
  • The run() method to do the work + for the action object
    • +
+ +

Instances of SessionAction are created + at the start of method compilation and destroyed at the + end of compilation. In other words, they are created + and destroyed every time when a pipeline with the + corresponding action is selected for method + compilation. Therefore, no data can be saved in this + object between compilation sessions.

+
+ +
IAction
+ +
+

The IAction class is a common ancestor + of the Action and + SessionAction classes, which contains + service methods that may be accessed from both child + classes; for example, argument processing methods or + methods for obtaining a reference to the corresponding + pipeline.

+
+
+ +

package

+ +

Figure 1: The Role of Pipelines in the JIT + Compilation Process

+ +

Figure 1 demonstrates the role of pipelines in the JIT + compilation process.

+

Back to Top

+ +

Implementation + Notes

+ +

When using the PMF implementation, please note the + following:

+ +
    +
  • The common pipeline is mandatory, others are + optional.
    • + +
  • An optional pipeline must be assigned to a unique and + non-empty filter expression.
    • + +
  • Different pipeline steps can refer to the same + action.
    • + +
  • Different pipelines can have different sets of steps + and different orders of steps.
    • + +
  • Each pipeline step can be supplied with a unique + parameter.
    • +
+ +

Back to Top

+ +

Pipeline Life-cycle

+ +

Initialization

+ +

The common pipeline is created at the JIT initialization. + All other pipelines are created just before their first use. + Therefore, an optional pipeline is only initialized when a + compiled method matches its filter.

+ +

During pipeline creation, the compiler iterates over all its + steps. For each step, the compiler creates an instance of the + referenced Action object and calls the + Action::init() method. At this point, all action + parameters are available and the Action::init() + method can use any of the IAction::getArg methods + to retrieve action parameters. For efficiency purposes, + parameters should be processed at the initialization phase, not + during compilation of each method.

+ +

Note

+ +

You can call the + IAction::getArg method from + SessionAction::run() during compilation of a + method, but it is a less effective way of parameter processing. + The SessionAction object has access to its parent + Action object and therefore, can access data + prepared during the initialization phase.

+ +

Method Compilation

+ +

At this stage, the compiler selects the pipeline for the + given method using the filter expression in the method filter + and proceeds to compile a method. If the method matches several + filters, then the pipeline with the longest filter expression + is selected, that is PMF always chooses the filter with more + characters in its expression. PMF prohibits creation of + different pipelines with the same filter expression.

+ +

To compile the method, JIT executes the selected pipeline: + iterates over all its steps, creates a + SessionAction object for each step, calls the + run() method in each object, and destroys it.

+ +

De-initialization

+ +

At the JIT de-initialization phase all created pipelines are + destroyed. The Action::deinit() method is called + for every step in a pipeline during its de-initialization.

+ +

Back to Top

+ +

Command-line + Interface

+ +

The command-line interface is based on system + properties defined by "property = value" relations. At the + initialization phase, all properties are stored in the table of + VM properties. Depending on the configuration mode, VM can + create several JITs. In this case, each JIT reads this table + and selects properties independently; therefore, different JITs + have different property sets.

+ +

The general form of the JIT command is the following:

+ 
+-XX:jit.<JIT>.<pipeline>.<parameter>=<value>
+
+ +

Where

+ +
    +
  • + -XX:jit. is the mandatory prefix enabling + JIT commands selection from VM commands. + +

    Note

    + +

    The Jitrino command-line interface + ignores all commands that do not start with this + prefix.

    +
    • + +
  • + <JIT> specifies the JIT name. + + +

    Note

    + +

    JITs are created by + Execution Manager (EM) using + information from the command line and/or + + execution manager configuration + file.

    +
    • + +
  • <pipeline> specifies the + pipeline name.
    • + +
  • <parameter> and + <value> are used to construct one + of the three available basic commands: + filter definition, + path definition and + argument + definition.
    • +
+ +

Both <JIT> and + <pipeline> elements define the + command scope in the following way:

+ +
    +
  • If the <JIT> element is + omitted, the scope of the command consists of all existing + JITs.
    • + +
  • If the <pipeline> element is + omitted, the scope of the command consists of all pipelines + existing in the specified JIT.
    • + +
  • If both <JIT> and + <pipeline> are omitted, the scope + of the command consists of all pipelines of all JITs.
    +
    • +
+ +

Commands can be specified on the command line or in the EM + configuration file. Most important commands are usually placed + in the + + EM configuration file, so Jitrino can be used + without any commands on the command line.

+ +

The order of commands on the command line and in the + external file is not significant.

+ +

Another way to deal with a set of commands is to store it in + a command file and read this file with a special read + command:

+ 
+-XX:jit.read=<command filename>
+
+ +

Note

+ +

Write commands in the command file without + the prefix -XX:jit. The read command + itself can be used in the command file. The command file can + contain empty lines and comment lines starting with + # or //.

+ +

For information on the general command line syntax + specification, refer to the + Appendix.

+ +

Back to Top

+ +

PMF Commands

+ +

To control the JIT compilation process, PMF uses + path definition, + filter definition and + argument definition + commands.

+ +

Path Definition

+ +

A pipeline can be described as an ordered set of terminal + leafs of a path tree. The tree is a purely abstract concept + introduced to describe the pipeline construction process. Each + non-terminal node of the tree is called a path and + defined by the following path definition command:

+ 
+-XX:jit.<JIT>.<pipeline>.path.<name>=<child1>,<child2>,...
+
+ +

The given command introduces a node with the specified name + and enumerates all its direct descendants - children. If a + child is a terminal node, it is treated as the name of the + action; if not, then the child must be another path.

+ +

For each pipeline, define one path with the + <name> element omitted, which is called the + root path. Each path must have a unique definition.

+ +

Each action in the pipeline has a unique full name + consisting of all node names from the root to the action + itself. The root node is excluded because it has no name. More + than one instance of the same action can exist, as long as each + instance has a unique full name.

+ +

Example

+ +

OPT is the JIT name + and a, b and + c are the action names.

+ 
+-XX:jit.OPT.path=M,N
+-XX:jit.OPT.path.M=a
+-XX:jit.OPT.path.N=b,c
+
+ +

The given commands describe the simple + path tree:

+ +

package

+ +

Figure 2: A Path Tree of a Common + Pipeline

+ +

The following resulting pipeline is + constructed from the aforementioned tree:
+ a, b, + c

+ +

The pipeline above is common, because it + has no method filter. Each action of the given pipeline has a + unique full name.

+ + + + + + + + + + + + + + + + + + + + + + + + + +
ActionFull Name
aM.a
bN.b
cN.c
+ +

Back to Top

+ +

Filter Definition

+ +

To define a filter, use the following command format:

+ 
+-XX:jit.<JIT>.<pipeline>.filter=<class>.<method><signature>
+
+ +

Where

+ +
    +
  • <pipeline> is the required + element, which cannot be omitted.
    • + +
  • <class> is the name of the + class constituting the filter expression.
    • + +
  • <method> is the name of the + method constituting the filter expression.
    • + +
  • <signature> is the signature + of the method constituting the filter expression.
    • +
+ +

Note

+ +

In the filter definition, the + method and the signature elements + have no period . between them.

+ +

You must follow the VM notation for the described elements. + You are free to omit some elements, but at least one must be + present.

+ +

If each of three elements of the method name starts with the + corresponding element of the filter expression, the compiled + method matches the filter.

+ +

Example

+ 
+-XX:jit.OPT.p.filter=java/lang/Thread.check
+
+ +

The following methods can match the + given filter:
+ java/lang/ThreadGroup.checkAccess()V
+ java/lang/Thread.checkGCWatermark()V
+ java/lang/Thread.checkAccess()V

+ +

To continue the example, methods can equally match a less + detailed filter expression, such as + java/lang.check. However, PMF would use the more + specific filter, as described in the + Method Compilation + section.

+ +

Example

+ 
+-XX:jit.OPT.path=M,N
+-XX:jit.OPT.path.M=a
+-XX:jit.OPT.path.N=b,c
+-XX:jit.OPT.p.path=M,N,O
+-XX:jit.OPT.p.path.O=a
+-XX:jit.OPT.p.filter=java/lang/Thread.check
+
+ +

The commands above are associated with + the following path trees:

+ +

package

+ +

Figure 3: An Example of Path Trees for + Multiple Pipelines

+ +

The following resulting pipelines are + constructed from the aforementioned trees:
+ (common) a, + b, c
+ (p)           + a, b, + c, a

+ +

Note

+ +

Path definitions M and + N in the pipeline p are + inherited from the common pipeline, because they were defined + in commands with the pipeline element omitted, but the root + path definition in the p pipeline was + overridden.

+ +

Back to Top

+ +

Argument Definition

+ +

To control the actions behavior, PMF supports an action + parameters passing convention. The parameter definition command + pattern is the following:

+ 
+-XX:jit.<JIT>.<pipeline>.arg.<full name>.<param>=<value>
+
+ +

Where

+ +
    +
  • <param> and <value> are + mandatory elements.
    • + +
  • <JIT>, <pipeline> and + <full name> are optional elements defining the + scope of the both.
    • +
+ +

The <full name> element defines the parameter + scope in the following way:

+ +
    +
  • If <full name> addresses a certain action, + that is a terminal node, the parameter affects all + instances of this action.
    • + +
  • If <full name> addresses not an action, + but a node with children, the parameter affects this node + and all its descendants.
    • + +
  • If <full name> is omitted, the parameter + affects all actions in the pipeline.
    • +
+ +

Example

+ 
+-XX:jit.OPT.p.arg.a.verify=1
+
+ +

The given command sets the + <verify> parameter for both a + instances, M.a and O.a, + to 1. See Figure 3.

+ +

The table below gives several examples of essential commands + for the path trees based on Figure 3.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
CommandOmitted ElementDefinition
+ -XX:jit.OPT.p.arg.N.b.verify=2 +
+ — +
+ 		The parameter verify + with value 2 is available only to action + b with the full name + N.b in pipeline + p of JIT + OPT.
+ -XX:jit.OPT.p.arg.N.verify=2 +
+ — +
+ 		The parameter verify + is available for actions b and + c simultaneously.
+ -XX:jit.OPT.p.arg.verify=2<full + name>The parameter verify + is available for all actions a, + b, c and + a in pipeline + p.
+ -XX:jit.OPT.arg.N.verify=2 + <pipeline>The parameter verify + is available for b and + c actions for common and + p pipelines of JIT + OPT .
+ +

The special form of argument definition commands controls + the pipeline construction process:

+ 
+-XX:jit.<JIT>.<pipeline>.arg.<full name>=on
+-XX:jit.<JIT>.<pipeline>.arg.<full name>=off
+
+ +

Note

+ +

In the given form of the argument + definition, the <param> element is absent and only + two values for <value> exist: on and + off.

+ +

The off value disables the addressed node and + all its descendants, while the on value enables + the node and its descendants. When constructed, a pipeline does + not include any disabled actions.

+ +

Example

+ +

To exclude action b + from the pipeline, apply the following command:

+ 
+-XX:jit.OPT.p.arg.N.b=off
+
+ +

To enable/disable nodes directly in the path definition + command, add state characters immediately after the node name, + videlicet:

+ + + + + + + + + + + + + + + + + + + + + + + + + +
CharacterCorresponding Node + Meaning
+
+ + +
+ 		enabled
+
+ — +
+ 		disabled
+
+ ! +
+ 		cannot be disabled
+ +

The following command states that c1 is + enabled, c2 is disabled and c3 is + enabled and mandatory:

+ 
+-XX:jit.<JIT>.<pipeline>.path.<name>=c1+,c2-,c3!
+
+ +

If you try to disable any mandatory node with the + off command, the fatal error occurs.

+ +

Back to Top

+ +

Logging System

+ +

The logging system supports diagnostic messages from + pipeline actions. The VM property mechanism fully controls the + system.

+ +

Defining Streams

+ +

A set of log streams is used for logging in Jitrino. + Each log stream has a name. The name of the log stream is not + related to the name of the file, to which the stream is + assigned. The class Jitrino::LogStream is the + program representation of a log stream. This class is similar + to the std::ostream class in the defining operator + << and other useful methods, such as + printf() and flush(). To obtain the + reference to the underlying std::ostream instance, + call the out() method.

+ +

Streams can be accessed by name or by ID. The stream ID can + be obtained from its name by using the + IAction::getLogStreamID() method. For performance + reasons, several most frequently used streams, so-called + known streams, have predefined IDs:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Stream NameStream IDOutput
infoLogStream::INFOThe protocol of compilation: JIT + and pipeline names, the method name and number, and so + on.
rtLogStream::RTRun-time output not related to + compiled methods.
ctLogStream::CTCompile-time diagnostic.
irdump + LogStream::IRDUMPThe dump of internal Jitrino + structures for a compiled method.
dotdump + LogStream::DOTDUMPThe dump of internal Jitrino + structures in the .dot format.
dbgLogStream::DBGDebug information.
+ +

To enable outputting log information to a stream, enter a + command of the following syntax:

+ 
+-XX:jit.<JIT>.<pipeline>.arg.<full name>.log=<stream1>,<stream2>,...
+
+ +

The left part of the command specifies the name of the JIT + compiler and the compilation pipeline to be used, whereas the + right part contains the names of the streams that you want to + enable. Use the same syntax to enable a custom stream: write an + arbitrary name for it and enable the stream with this + command.

+ +

Note

+ +

If program output is directed to a disabled + stream, the output is lost and no error diagnostic is + produced.

+ +

The enabling command follows the general scope of rules: the + stream is enabled for the node specified in the command and all + its descendants.

+ +

Streams defined for different nodes are different streams, + though they can have the same name.

+ +

Example

+ +

The following commands enable two + different streams, though both can be accessed from actions + N.b and N.c by the same + name. The streams can be assigned to different files.

+ 
+-XX:jit.OPT.p.arg.N.b.log=ct
+-XX:jit.OPT.p.arg.N.c.log=ct
+
+ +

Back to Top

+ +

Using Filters in Logging

+ +

A method filter can be used to filter + logging by package, class or method name. For logging purposes, define an empty + pipeline with an appropriate method filter, with this pipeline + considered a filter name for simplicity.

+ +

Example

+

The following commands implicitly define + an empty pipeline f for filtering purposes, + defines a method filter for this pipeline and enables logging for + the methods corresponding to f.

+ + 
-XX:jit.OPT.f.filter=main
+-XX:jit.f.arg.log=ct
+
+ +

Back to Top

+ +

Assigning a + Stream to a File

+ +

To assign a stream to a file, use the following command + syntax:

+ 
+-XX:jit.<JIT>.<pipeline>.arg.<full name>.log.<stream>.file=<filename mask>
+
+ +

In the given command, the <filename mask> + element constructs a filename for the stream. All characters + from this mask are copied to the resulting filename literally + except macros, which are expanded and the result of expansion + is copied to the filename. Below is the list of available + macros.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
MacroExpands To
%jit%The name of the JIT that is + compiling the method.
%class%The method class name with + / changed to _
%class_tree%The method class name with + / preserved.
%method%The method name and signature in + VM notation.
%seqnb%The global sequential number of + the current compilation.
%log%The name of the stream.
%thread%The number of the thread + compiling the method.
+ +

In most cases, a program can access the stream file from + concurrent threads, so that all stream operations are + synchronized. However, if a filename mask for the stream + contains the %thread% macro, this file is + thread-specific and access to it is not synchronized.

+ +

Using Filename + Masks

+ +

For all streams, the following default filename mask is + used:

+ 
+log/%jit%/%class%/%method%/%log%.log
+
+ +

For example, for an enabled stream abc with no + assigned filename mask, the log file is in the same directory + with other known stream files, and its name is + abc.log.

+ +

All known streams have hard-coded default filename masks, + described in the table below. These masks are used if no + assignment is made with the corresponding command.

+ + + + + [... 176 lines stripped ...]
Stream Name