ant-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eh...@employease.com
Subject RE: Working around jikes bug in large projects
Date Tue, 27 Aug 2002 21:10:51 GMT
In response to Mr. Duffey, yes, this would be when someone does a clean
build, where none of the class files exist.  Our source is laid out in over
230 packages, and the "common" packages segregated from the individual
application modules (i.e., logical, functional subsections of the overall
application as opposed to separate applications).  The common packages have
no dependencies on the application modules, and except for a few ugly
exceptions, application modules don't have interdependencies.  However,
these separations between modules are strictly logical - we don't jar up the
modules individually.  This is fine for us, since we only have a single
application and don't have to share, for instance, the common packages
across multiple applications or projects.

The reason jikes chokes on a large number of classes is that instead of
limiting the number of constants for a single class file to 64K, it's
enforcing that limit across all the classes passed to it.  (At least this is
my understanding from the bug report.)  It's not the number of classes, but
rather the number of constants defined by those classes.  Jikes doesn't
error out, unfortunately, and generates bad class files.  Our overall
application certainly exceeds that limit, so doing a clean build results in
corrupted class files.  When I do a build where only a few source code
modifications have been made, only the modified files are compiled and the
class files are fine.

Based on the discussion so far, I seem to have the following options:
- Use javac instead of jikes.  I'd rather avoid this for political and other
reasons.
- Rather than use a single <javac> task, break it up into several <javac>
tasks, and with proper ordering based on dependencies, I can reduce the
number of classes that actually get compiled due to dependencies.  My
concern here is that due to the nature of the bug, I have no way of knowing
if the subset of classes I've chosen to compile in a single <javac> task
exceeds 64K constants.  Worse yet, someone could add constants to a class
(or even add a new class) that would cause the number of constants to exceed
64K, despite my initial, careful decomposition of the build into multiple
<javac> tasks.

Ed

-----Original Message-----
From: Duffey, Kevin [mailto:KDuffey@BUYMEDIA.com]
Sent: Tuesday, August 27, 2002 3:31 PM
To: Ant Users List
Subject: RE: Working around jikes bug in large projects


To the original poster, is this like a nightly build, where you erase
all compiled classes, then do a fresh build? Or is this the process you
do every time? We used to build ~2000 files and the very first build
would take about 30 seconds. After that, it would take less than 1
second to build a few files. I used a single ant script, but our /src
dirs were separated into functional areas. We had a core, then specific
modules. I suppose it depends on the project, and I don't think the
original poster really clarified enough on how the structure of the
source is laid out. I am hoping packages and dirs are used. Maybe if the
original poster can enlighten us as to if this is the build step he
takes every time he makes a change, does it compile all classes, and so
on? I still see no reason why any compiler would choke on any number of
classes. It may run out of memory figuring out all the dependencies, but
I have often seen javac not catch already-compiled class dependencies.
For example, I compile one class, it depends on another. I later change
that other class in such a way that the first compiled class would also
have to be recompiled, because lets say a method it uses is no longer
present. Thing is, it NEVER recompiles the first class. The only thing I
EVER see compile are classes that have the .java source changed, that is
it. Heck, maybe there is something I am doing wrong? I would think
anytime a class/.java src changes and it affects another class, the
compiler would also recompile that class even if it's source did not
change. I have never seen this happen though, so if I am doing something
wrong, perhaps a setting I need to pass, by all means please let me
know.


-----Original Message-----
From: Alexey Solofnenko [mailto:A.Solofnenko@mdl.com] 
Sent: Tuesday, August 27, 2002 12:11 PM
To: 'Ant Users List'
Subject: RE: Working around jikes bug in large projects

While you are right from organizational point of view, there are some
problems. Let consider the following scenario:
- you have updated and compiled the whole project on your computer;
- you changed a class in a component that every other component is
using;
- before checking in you want to check whether it at least compiles and
it
does not break other components;
+ the component compiles correctly;
+ while compiling other components javac does not care that files on
classpath are changed, so it does not recompile the component;
= you do not know whether the code you have changed really works

So the only way to ensure that everything is fine is to perform a clean
build. By executing one javac, there is no problem.

- Alexey.

-----Original Message-----
From: Dominique Devienne [mailto:DDevienne@lgc.com] 
Sent: Tuesday, August 27, 2002 11:59 AM
To: 'Ant Users List'
Subject: RE: Working around jikes bug in large projects

Javac doesn't perform a full dependency analysis on the *sources* it
compiles. My point is that when you compile the sources of a module
(which
are in a source tree of their own), the modules it depends on are
referenced
as JARs (or classes dirs) on the <javac> classpath. The source of the
module
can never be out of sync it's depend JARs (and their own corresponding
sources).

You can still have issues like 2 modules depending on the same third
low-level module using incompatible versions of that modules... Stuff
that
leads to runtime errors like method not found and co. This is easily
avoidable when you control all the sources, and can setup your system to
compile projects in the right order, from low level to higher levels.
You
can even setup a Gump build, like the Jakarta one, to make sure all the
modules are all compatible.

So yes, keeping everything in a single project with a single <javac>
works,
but it doesn't scale, and doesn't usually lead to clean projects,
composed
of lean and mean modules, with clear and defined (and few) dependencies
between them. The single project architecture is easier to setup, but if
your project keeps growing, and your code gets re-used by other people,
then
it's bound to bite you in the end... It's biting us for sure! --DD

-----Original Message-----
From: Alexey Solofnenko [mailto:A.Solofnenko@mdl.com] 
Sent: Tuesday, August 27, 2002 1:44 PM
To: 'Ant Users List'
Subject: RE: Working around jikes bug in large projects

Are you sure, that javac correctly tracks classpath/library changes? I
had a
few encounters that I can explain only by javac not checking that files
on
classpath were changed. We perform clean before javac if it happens.

- Alexey.

-----Original Message-----
From: Dominique Devienne [mailto:DDevienne@lgc.com] 
Sent: Tuesday, August 27, 2002 11:38 AM
To: 'Ant Users List'
Subject: RE: Working around jikes bug in large projects

Nope! Just put the source for the different modules in separate source
directories, so Javac can only find classes this module depends on
thanks to
the classpath, and not through any sourcepath, and make sure to use a
clean
classpath. You do not need to humongous single <javac> to ensure correct
dependencies!!! --DD

-----Original Message-----
From: Alexey Solofnenko [mailto:A.Solofnenko@mdl.com] 
Sent: Tuesday, August 27, 2002 1:19 PM
To: 'Ant Users List'
Subject: RE: Working around jikes bug in large projects

However a single command compilation is almost always faster. At least
it is
the only way to be sure that all dependencies will be correct.

- Alexey.

--
{ http://trelony.cjb.net/   } Alexey N. Solofnenko
{ http://www.inventigo.com/ } Inventigo LLC
Pleasant Hill, CA (GMT-8 usually)


-----Original Message-----
From: Dominique Devienne [mailto:DDevienne@lgc.com] 
Sent: Tuesday, August 27, 2002 11:10 AM
To: 'Ant Users List'
Subject: RE: Working around jikes bug in large projects

Just increase the memory for <javac>, or better yet, compartiment your
software into independent modules that can be compiled in the right
order
independently. Will makes you software better for it, and it's always a
very
good idea to have clean separation (and clear dependencies) lines
between
your different modules (I can't imagine a single module of 3000 files!).

We used to compile ~5500 files in one pass (needs gobs of memory!), now
it's
two independent ones of ~2500 and ~3000 files, and we're breaking that
down
further to 6/8 different projects. --DD

-----Original Message-----
From: ehowe@employease.com [mailto:ehowe@employease.com] 
Sent: Tuesday, August 27, 2002 12:59 PM
To: ant-user@jakarta.apache.org
Subject: Working around jikes bug in large projects

Has anyone run into jikes bug #222 (compiling large # of classes leads
to
ClassFormatErrors) when using ant/jikes for large projects?  (For
information on the bug, see
http://www-124.ibm.com/developerworks/bugs/?func=detailbug&bug_id=222&gr
oup_
id=10)

I'm in the process of changing from makefiles to ant for our build,
which
consists of over 3000 java files.  Our makefiles call jikes for each
java
file, so we have never run into this before.  We also have a makefile in
each package.

With ant, I currently have a single build.xml at the top level of the
project tree, which basically ends up feeding all 3000 files to jikes at
once, exposing this ugly jikes bug.  I've tried switching to javac, but
I
run out of memory trying to compile it.  In any case, I'd rather stick
with
jikes if at all possible.

Has anybody else run into this?  How did you work around it?  I suppose
I
could put a build.xml in each package, but that still wouldn't guarantee
I
wouldn't run into this bug again.  Besides, isn't that counter to ant
philosophy?

Thanks,
Ed

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  Ed Howe * Software Architect * Employease, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



--
To unsubscribe, e-mail:
<mailto:ant-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:ant-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail:
<mailto:ant-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:ant-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail:
<mailto:ant-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:ant-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail:
<mailto:ant-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:ant-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail:
<mailto:ant-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:ant-user-help@jakarta.apache.org>

--
To unsubscribe, e-mail:
<mailto:ant-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail:
<mailto:ant-user-help@jakarta.apache.org>


--
To unsubscribe, e-mail:   <mailto:ant-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:ant-user-help@jakarta.apache.org>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message