harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Etienne Gagnon <egag...@sablevm.org>
Subject Re: [classlib] Preprocessor - CHECKPOINT
Date Tue, 31 Oct 2006 18:55:04 GMT
[WARNING:  Long!]

Here's an idea.

It would be quite easy (for me, at least ;-) ) to develop a
"line-preserving" AND "revertible" processing tool so that:

1- source + javac => j2se

2- source + processor(target) + javac => target,  where target is a
   parameter (j2me, etc.)

3- source + processor(target) => source for target, where line numbers
are identical to source

4- source for target + reverseprocessor(target) -> source :-)  That's
the thing lacking in most preprocessors, as often you would like to
modify the processed source and propagate these changes back to the
initial source; this is usually a pain as it must be done manually.  Not

I am, usually, in favor of using branches, for development, yet, in the
current case, I think that branches have their drawbacks:

1- I have yet to see a branch/merge system that deals well with major
code reorganization in a single file.

2- There's always the infrequent, yet very annoying (if not dangerous
when it happens) semantically invalid merge result.

3- Developers of the main branch will remain mostly unaware of the
parallel development on other branches.  In this case, this can be a
problem, as having j2se developers aware that they are changing lines
for which other branches maintain distinct versions ones might be a good
"communication tool".

The idea, for line preservation, would be to either use "revertible"
comments (easiest) or maybe play with special Java annotations, if they
prove sufficient for the task.

The advantage of "intelligent" Java-syntax-aware processing (instead of
dumb preprocessing) is that the processor could work intelligently.

Here's a tentative example to illustrate my idea.  (This is not a fully
thought-up example/syntax; it's just a way of trying to explain the

Java source  (j2se)


//  @Processor(add to j2me
//  float some_field;
//  )

  @Processor(Not in j2me!)
  int some_field =
    some +


Processed source  (target => j2me)


  @BeginRevert(add to j2me)

  float some_field;


//  @Revert(@Processor(Not in j2me!)
//  int some_field =
//    some +
//      initializing()
//        code; )


Modified processed source ;-)


  @BeginRevert(add to j2me)

  float some_field = missing initializer!;


//  @Revert(@Processor(Not in j2me!)
//  int some_field =
//    some +
//      initializing()
//        code; )


Revert-processed source (=> ready for "svn ci")


//  @Processor(add to j2me)
//  float some_field = missing initializer!;
//  )

  @Processor(Not in j2me!)
  int some_field =
    some +


The "intelligent" part comes with the idea that the @Processor
directive, in the j2se uncommented source above, "knows" that it applies
to the whole field declaration, including any initializer, regardless of
the number of lines used by the field declaration.  Such directives
could be "limited" to work on:
1- classes/interface declarations
2- method/field declarations
3- static statement blocks
4- statement blocks in general
5- other things?

The idea is that it is not a "character/lined based" processor, but
really a syntax-driven one.

Using comments (as above) is optional, yet I recommend doing it so that
text editors don't syntax color unapplied code in any current situation
(source, processed-source).

Both j2se & j2me (as well as other target developers) will be able to
work on the "processed" and the original source code.  Typically, the
revert processor will be used by j2me developers before "svn ci" of
their modifications.

It would be quite easy for me to develop such a tool using SableCC.
FYI, the output of SableCC is not encumbered by any copyright other than
whatever the "grammar" developer wants.  Also, I will contribute the
developed tool to Harmony if this approach is chosen (or develop it as a
Harmony developer, if you prefer...).
[Geir: Yes, I know, I have to send the paperwork...]

Some advantages:

1- Even if you package "processed source" into a jar file (for j2me
users), line numbers match exactly.  Bug reports from users won't be
about Mars or Jupiter. :-)

2- Debugging using both initial and processed source works flawlessly.

3- Major code reorganization can be done cleanly without the need for
bug-prone, tricky, hand-made merges (as automated merge can be
misleading in such case)

4- We can make the tool and its syntax as "gentle" as you want; we have
total control on its syntax and semantics. :-)

5- We could add a "syntax checking" mode to the processor, so that in
that mode, *all* code would be checked for syntax correctness (even
commented processor directives)  instead of modified for a specific
target.  This could be added to the ant script for all developers, so
that at no point somebody "svn ci" syntactically invalid code in case
they, by error, changed some of the commented directives...

6- We could even implement a "not in processed form" check mode that
could be added as a pre-commit check phase, so that we make sure code is
never checked-in in processed form.

Anyway, that's an idea you could play with.

Have fun!


Geir Magnusson Jr. wrote:
> So we all agree that it's not an ideal solution.
> Can anyone think of anything else?  No one said this was going to be
> easy...

Etienne M. Gagnon, Ph.D.            http://www.info2.uqam.ca/~egagnon/
SableVM:                                       http://www.sablevm.org/
SableCC:                                       http://www.sablecc.org/

View raw message