harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Blewitt" <alex.blew...@gmail.com>
Subject Re: Re: [classlib][pack200] new module (was: [classlib][pack200] Development has stalled :-( )
Date Wed, 10 Jan 2007 22:21:38 GMT
On 10/01/07, Tim Ellison <t.p.ellison@gmail.com> wrote:
> The working model for contributing code.
>
> Taking them in reverse order, there have been a number of suggestions
> for working 'disconnected' and contributing patches / new files from
> your local repository (such as svk and git) that I hope you can figure
> out a system that works for you.  It is a problem that all contributors
> face, and a great topic for sharing experiences.

Yup. And realistically, I've been frustrated by a few things (like the
fact that binaries aren't included in diffs; we should make that
clearer in the doc) as well as having a tendency to wait until patches
are applied. It mostly seems to be the new files, but it's still
growth rather than patching existing files for the most part now.

Having some kind of intermediary (e.g. the git or whatever previously
discussed) seems like an idea, but ideally I want to get decent
support in the Eclipse IDE as opposed to doing things on the command
line. Subclipse has that support, so there's an argument for staying
with that.

> Additional requirements on the Pack200 code, and
>
> IMO the additional requirements on the Pack200 code seems to
> unnecessarily couple implementation with packaging/deployment needs.  I
> don't think making Pack200 a separate bundle will be sufficient to meet
> all the requirements.  Let me try and explain why:
>
> - dependency on Java 1.4 syntax and APIs
>
> We can easily ensure that the Pack200 code only uses 1.4 syntax by
> compiling it with the appropriate source flag.  However, we cannot
> ensure it only uses 1.4 APIs since we compile Harmony against itself.
> The only way to do that is to have a JSE1.4 class library around, which
> means an additional Harmony dependency.

Whilst you can't depend on those APIs, I can, because I'm writing on a
system which has a JVM 1.4 on it. And it's important to me to make
that happen. The automated build can only check for 1.4 source
compatibility; but that's still important, because I'm compiling
against a 1.4 VM that can't handle 1.5 stuff.

> I suggest that we compile the code in a separate source folder as 1.4,
> but rely upon an out of band build to check that there are no references
> outside the J2SE-1.4 execution environment.

You *have* to have that in the IDE too -- it's not sufficient to have
it in the build. That's what I've set up in the pack200 project (as
per the patch recently submitted).

> If people commit 5.0 syntax, or code that depends upon 5.0 APIs then it
> is not your job to fix it -- you should yell on the dev list and have it
> rolled back.

The point is this can be done inadvertently. *every* *time* I have
submitted a patch, and then updated later, it's been polluted with 5.0
syntax. In the last case -- the package refactoring -- it was because
someone had run a 'Clean Up' on a few source files, and because the
project was set to 5.0 compatibility. That would not have happened --
or been possible -- if the source code was set to 1.4 compatibility.

You *cannot* configure Eclipse to have a project that contains a
source folder for 1.4 and a source folder for 1.5. They must be
separate projects. This is, in effect, what we have now. In fact, I'd
argue that we should have a separate project for the pack200-java5
code; but the chances of me convincing you that I should add another
project (despite being the right thing to do) are slim to
non-existant. So, I've compiled/uploaded a .jar with 1.5, and put it
on the classpath, so that if you're on a 1.5 system you can take
advantage of it. On a 1.4 system they won't load -- but that's OK,
because the interface they implement was only added in 1.5 anyway, so
1.4 users won't use those classes.

The build.xml correctly builds the source files with a 1.4 target (for
the src/main/java) and a 1.5 target (src/main/java5) files each time.

> - packaging as a separate bundle
>
> In regular OSGi you do not explicitly list java.* packages in the
> Import-Package statement.  They are part of the execution environment
> hosting the bundle.  However, since Harmony is modularizing the class
> libraries themselves we precisely want to define the scope of the bundle
> by java.* and their corresponding implementation packages.
>
> For the Pack200 code to be a useful bundle in Harmony will require a
> different Manifest to a bundle that is useful as a regular Eclipse
> plug-in / Felix bundle / whatever.  Harmony's bundles cannot specify a
> required execution environment.  One manifest will not do both jobs.

You are incorrect. I have the bundle running in an Eclipse instance right now:

osgi> install file:///tmp/pack200.jar
Bundle id is 244

osgi> start 244

osgi> ss
Framework is launched.

id      State       Bundle
0       ACTIVE      system.bundle_3.2.0.v20060510
1       ACTIVE      org.eclipse.equinox.common_3.2.0.v20060512
2       ACTIVE      org.eclipse.update.configurator_3.2.0.v20060511
:
238     RESOLVED    org.tigris.subversion.subclipse_1.0.3
242     RESOLVED    org.eclipse.platform.doc.isv_3.2.1.200701082312
244     ACTIVE      org.apache.harmony.pack200_1.0.0

This is using the same manifest that's in the code at the moment. All
that's needed is to make the resolution of the Import-Package
optional:

Manifest-Version: 1.0
Specification-Title: Java Platform API Specification
Specification-Version: 1.5
Implementation-Title: Apache Harmony
Implementation-Vendor: The Apache Software Foundation
Implementation-Vendor-Id: org.apache.harmony
Implementation-URL: http://harmony.apache.org
Bundle-ManifestVersion: 2
Bundle-Name: Harmony Pack200
Bundle-SymbolicName: org.apache.harmony.pack200
Bundle-Version: 1.0.0
Bundle-ClassPath: .
Eclipse-JREBundle: true
Import-Package: java.io;resolution:=optional,
 java.lang;resolution:=optional,
 java.util;resolution:=optional,
 java.util.jar;resolution:=optional
Export-Package: org.apache.harmony.pack200,
 org.apache.harmony.pack200.bytecode
Bundle-RequiredExecutionEnvironment: J2SE-1.4,
 J2SE-1.5

On the proviso that the manifest parsing is following the OSGi syntax
in Harmony correctly, this will perform the correct binding when
present in an environment where those packages are exported; and in
the case where they are not, will work just the same.

Let's be clear: I started this work so that it would be available as a
separate bundle. The pack200 that I've installed is the output of the
build.xml; the intention was always to have the same bundle running in
both Harmony and Equinox (or other OSGi bundles). After all, isn't
this what OSGi bundles/modular architecture is all about?

> I suggest that there needs to be a separate project description and
> Manifest for a 'regular' Pack200 bundle outside the modules source tree,
> that links to the same source folder containing the same implementation
> code.  Then there can be a 'Harmony view' and 'regular bundle view' of
> the same code.

They are one and the same, and there appears to be no problems with
this being the case. Perhaps someone on a proper Harmony supported
platform (i.e. not a Mac :-) could verify that the above manifest
works for the pack200 still works when running the tests under
Harmony? If that's the case, I see no reason why we should need a
'Harmony view' and 'regular bundle view' distinction.

> - NLS messages
>
> Sure, if the code is going to be separable, then it needs a way to take
> only it's own messages away.  The mechanism for loading messages is
> lightweight so I suggest there is no need to re-invent that code.

Agreed.

> So then, does this warrant a new module, and what is a module anyway?
> Why is 'regex' a module, etc?
>
> The original discovery of modules was driven by the observed coupling in
> the APIs and implementation of the class library code.  We split up
> groups of functionally coherent behaviour, and declared them a module.
>
> Later we decided that we would not split along the API's SPI boundaries
> too, so, for example, the LDAP and DNS providers are in the JNDI module,
> and the SHA1, PKCS code are in SECURITY module etc.

Why not? It would have seemed a sensible decision to make. But those
decisions have already been made.

> I see the Pack200 being similar.  It is a provider to the ARCHIVE
> module, and by the same reasoning would be part of that module.  The
> additional requirements means that I'd ensure it was in a separate
> source folder, and a non-Harmony module project made to deliver that
> code into a regular OSGi bundle for consumption elsewhere.

I've already said that it is not going to work in a single Eclipse
project. The build assumptions are that (a) each module is its own
project, and (b) they're all at the same level. In any case, Eclipse
is not well suited to nested project structures like that.

In any case, what you see and I see are different things. I see the
pack200 implementation as being something that's usable by Java
applications, other OSGi platforms, and Harmony. Polluting/bloating
that with java.util.jar code destroys that view of the world. Perhaps,
since you believed that it's necessary to have two separate bundles
that another target would make sense. Given that it's not necessary,
and that 1 module == 1 bundle and the Harmony Pack200 Bundle == OSGi
Pack200 Bundle, I really don't see the desire to prevent this from
happening because of other decisions made in the past.

> Could we have made a different choice about projects?  Sure.  We could
> pick finer-grained bundles and pull out all the reusable pieces and
> providers into projects/bundles.  I can imagine having at least 3x as
> many modules for Harmony [aside: I've always wanted separate test code
> projects].

I'd argue that this is a good goal to work towards. Indeed, that's
what I'm encouraging for the pack200 module. So why, then, are you
against the idea? Shouldn't this be a goal to aim for for new modules,
and/or look at trying to make it happen for old modules?

> That is how we got to this set of modules and their 'semantic size'.
> Could we, or should we, have done it differently?  Maybe, it has worked
> well for us so far in terms of resolving multiple contributions, and
> switching different implementations (was it 3 RMI's at one point?!).

Let's distinguish between 'what was done' with existing projects, and
'what we want to do with pack200'. I made my views known when I
started contributing code, that I wanted it as a separate module, and
that it had to run on 1.4. I'm making that happen. The solution we now
have works, and works well -- the same deployable bundle runs under
Eclipse and (presumably) Harmony too.

OTOH the only reason for moving the pack200 back into the archive
module seems to be historical. The solution won't work -- 1.5 code
will be checked in, as it has repeatedly in the past -- and the
generated code (with java.util.jar stuff in) is extra bloat that's not
required for the general OSGi case (and in fact, may well have
problems). Not only that, but the modules are supposed to be, well,
modular and the finer grain the better. As I've noted, it would make a
whole lotta sense to have another pack200-java5 project rather than
bodging a checked in Jar file in the project.

> But like all parents, you know that if you are asked 'why?' enough times
> then the answer will eventually be 'because!' <g>.  If we want to
> revisit this decision then fine, but let's do so consistently for the
> whole project, and not build a patchwork architecture of different local
> decisions.

Right, but it's also important to consider things on a case-by-case
basis rather than relying on dogma.

It's also the case that the SPIs for Harmony (e.g. the JNDI, DNS) are
currently not planned to be reused outside of the Harmony framework.
If they were, perhaps the idea of splitting them out too would make a
lot more sense. However, if they aren't being used then there are less
benefits in doing so.

The fact that one example isn't going to be used outside of Harmony
doesn't imply that another example is also not going to be used
outside of Harmony. Therefore, straitjackets around the one that *is*
going to be used outside of Harmony just because there's one that
isn't is not going to help anyone.

> I've seen good-will on both sides to try and get a workable solution.
> Let's keep talking until we get a satisfactory resolution.  And in the
> meantime,keep hacking on the code :-)

There's three outcomes:

1) The code moves back into the archive module, where it was before.
Multiple source folders in one project (with 1.5 compatibility) are
created. People randomly check in 1.5 annotations during automated
Clean Up operations. It no longer works on pre 1.5 systems. Harmony is
happy, but I'm not.
2) The code stays in the pack200 module (either with or without a
separate pack200-jav a project). I'm happy, and the bundle works in
both Harmony and other OSGi systems with no changes. Harmony gets a
pack200 bundle that is integrated with the build and it lives in the
correct upstream project.
3) The code moves off to eclipse.org, where I develop it as an OSGi
bundle, either under EPL or dual EPL+Apache License. I'm happy, but
Harmony is not. Whilst there's no reason you couldn't hook it up as an
external dependency for download and weave it into the build (assuming
AL, of course)

I've argued that the 2) is the middle ground, keeping everyone happy,
and I've gone to great lengths to point out why 1) won't work. And,
other than 'we didn't start doing it that way', I believe that I've
addressed all the points you made.

Realistically, it's not my call to make. I don't think I can make any
more persuasive arguments than I already have done, so the decision is
up to the Harmony community. I'll keep working on the code in the new
pack200 location (assuming that
http://issues.apache.org/jira/browse/HARMONY-2957 gets applied). But
I'd like to get a final answer whether we're going with 1), 2) or 3)
here, because I don't want the code to move under my feet whilst I'm
trying to create patches -- the merge would be a nightmare.

Alex.

Mime
View raw message