harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geir Magnusson Jr <g...@pobox.com>
Subject Re: Supporting working on a single module?
Date Wed, 10 May 2006 12:07:28 GMT
Mark Hindess wrote:
> On 9 May 2006 at 10:32, Geir Magnusson Jr <geir@pobox.com> wrote:
>> Mark Hindess wrote:
>>> As the Harmony Classlib grows, I think that being able to work on a
>>> single module (or some subset of the modules) will become the
>>> typical way of working for many (perhaps even most) contributors.
>> Sure, that makes sense.
>>
>>> So I think we need a plan to support this.  I also think that
>>> forcing ourselves to support this mode of working sooner rather than
>>> later will help us keep from accidentally breaking the modularity in
>>> the current build/development process.
>> Can you not work on a single module now?
> 
> Yes.  But only by checking out everything.

Wouldn't you do that anyway?

I guess thats the mystery to me.  I figured that someone would do the 
full checkout, and go from there.

> 
>>> Oliver is currently looking at restructuring the native source to
>>> the appropriate modules/*/src/main/native directory.
>> Can he talk about it out here,
> 
> He/We did - more than once.  For instance:
> 
>   http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200603.mbox/%3c4410328E.4070904@googlemail.com%3e
> 
> There are quite a lot of issues when you think about how to get from
> where we are today to where we want to be.  It's not as trivial as it
> sounds so even writing a good description of the issue(s) for the list
> is not straightforward.

There are few alternatives though in an Apache project.

> 
>> or is this for your internal use?
> 
> Definitely not.  We sometimes talk to each other off-list but not very
> often!  ;-)
> 
>>> One question that comes out of this investigation is: where should
>>> the include files "live"?  I think they belong with the module that
>>> provides the implementation defined by the declarations in the
>>> header.  That is, zipsup.h is "owned" by archive, hythread.h is
>>> "owned" by thread (luni), etc.
> 
>> Sure - that makes sense.
> 
>>> However, if the build was to refer to the header files within the
>>> modules then this breaks the modularity.  So, for example, someone
>>> working on a single module would have to checkout all the dependent
>>> modules rather than just the one they wish to work on.
>>>
>>> So, perhaps, at build time we should copy the header files out of
>>> the owning module into a common include directory.  All modules
>>> would then refer to the header file in the common include directory.
> 
> You didn't comment on this bit.

No - I had something, and removed it so I could think about it.  In past 
lives, I've done a lot of C and C++, so was trying to recall any tricks 
we had.

I didn't remember any.  I suppose this is an example of the downside of 
working in glorified assembler or glorified assembler with bolt-on OO 
extensions. :)

> 
> I don't think it's a good idea for one module referencing into another
> directly where it isn't a well-defined interface that we can
> manage/enforce appropriately.  Would you agree?

But I don't see a solution here, yet.  Just balling things up into a 
"HDK" doesn't seem to fix it.

> 
> We might as well try to agree how to construct and manage this
> "interface".  I think the copying/"hdk" idea is a good solution.

I guess I don't grok how the copying solves the root problem of 
coupling, because the cross-coupling due to C/C++ is still there.

I certainly can see that the copying is useful - for a given platform 
target, you can copy the stuff to the top, I guess.

But I can also see having a "top" level per module, for which the target 
platform is copied into from the bowels of the module...

That seems cleaner and keeps separate "namespaces".

> 
> I didn't imagine these issues would be this contentious or perhaps I'd
> have tried to separate these two (related) issues.

That may be the problem here - I'm confusing the two issues?

> 
>>> This means we can then create a snapshot tar/zip of the common
>>> include directory which someone wishing to work on a single module
>>> could download to build against.  This is not dissimilar to the
>>> current way in which you could build the java sources against a
>>> deploy/jre snapshot.
> 
>> Why wouldn't someone wishing to work on a single module just checkout
>> all the code?
> 
> So someone working on prefs (which is approximately 2MB) would need to
> check out all the source for luni (the current largest module at ~36MB),
> awt, swing, sound, etc. ?

Yes.  I actually think they would if it builds.  It appears to me that 
the classlibrary is intercoupled - our modules are an unnatural (to the 
status quo) segmentation we've placed on top in our quest for something 
better.

I guess I need to re-read and figure out how this would solve that 
problem.  It seems to just add to the number of moving parts.

> 
> I usually have half a dozen workspaces where I'm trying things out (even
> more at the moment since I'm looking at the four contributions that are
> being voted on).  This isn't too bad at the moment with each one being
> about 1/4 GB but it will get bigger over time and therefore less
> manageable.

Aha...
> 
>> I'm really wary about having non-SVN-sourced artifacts meant to be
>> used in building/development.
> 
> Isn't that modularity?  Why shouldn't we do it - our customers will be?

Because our users will be using released things.  Things that are 
stable.  A "*DK" by definition is stable, stable code, stable 
interfaces, etc.  (ok, slow moving, really, but you know what I mean...)

> 
>> Smells a bit like the first step towards the classic Windows
>> "dll-hell".
> 
> Undoubtedly, with people working on separate modules, we will get build
> breaks.  But we'll get them when, for example, we don't have sufficient
> unit test coverage within the module being worked on - we had an example
> of this not long ago IIRC.  We'll also have breaks when people have made
> different assumptions about the meaning of the spec or the definition of
> the internal API.  It is a good thing that we find these *bugs* within
> the project... being our own customers!

I don't understand how a "hdk" helps find the bugs.

> 
> I think this will actually help *avoid* some of the problems you are
> thinking about.
> 
>>> For windows, the snapshot would also include the .lib files that are
>>> required to build for the run-time .dll files.
>>>
>>> What is this new snapshot?  Well, Tim suggested 
>> (Where?)
>>
>>> that it was a little like the jdk but for Harmony development.  So
>>> perhaps it should be called an hdk, Harmony Development Kit?
>> I'm missing the point... why wouldn't a developer checkout head, build
>> that, and then get to work on whatever part of whatever module they
>> wanted?
> 
> The classlib is getting pretty big.  If we are serious about modularity,
> then we should try to support it right through from build, development,
> deployment and ultimately at runtime.

I think that we're serious about modularity as a packaging option (our 
primary one) and something we're going to push for in the ecosystem. 
However, I'm afraid of letting the tail wag the dog here...

[SNIP]

> 
>> Is it that since modules reference other modules, working in one
>> module means you need the dependent modules around?
> 
> Yes, exactly.  We shouldn't require users to have the entire "source"
> for all the dependent modules around. 

Are you accidentally conflating "user" and "developer" here?    They are 
entirely different roles.

Also, I certainly can see the theoretical value of this, but we tried 
similar things in Geronimo, and it was, well, hell, because of the 
coupling between Geronimo and it's dependencies.  eventually, we gave up 
and had an 'uberbuild', which wasn't so bad - you'd checkout everything, 
do the master build, and then go dive into your module, which is what I 
think talked about doing here before.

So I guess I'm just resistive due to painful experience on this.


> We should support modularity at
> the development level.  (Like we do already with the stubs for the
> luni/security kernel classes.  We don't require developers to have the
> VM around.)
> 
> When I'm developing C code I reference the libc header files but I don't
> go poking around including random headers from the libc source.
> 
> So if I'm working on sql, I don't see why I shouldn't develop using
> header files and other well-defined parts of the API for other modules
> rather than having to have all of the source code checked out.

Because I think version problems are going to spiral out of control.

> 
> Of course, we should still support people working by checking out
> everything but we shouldn't require it.

This has to be the case.  And IMO, whether you did something like "svn 
co; build top" which fills in the "top-level header directory" (which I 
think might be better per module), builds all jars, test jars, and libs, 
and puts everything in place so you can go wander off and work in a 
module, or do a lightweight checkout, stuff the hdk in a specific place 
in your classlib/trunk tree, apart for versions, the situation must be 
*identical*, or you are going to run into all sorts of build and 
debugging/discussion hell.

> 
>> A long time ago we talked about something like this, a pre-build step
>> that produces a "hdk"-like set of artifacts that a developer could
>> then use if they were focused down inside a module.
>>
>> Is this the same thing returning for discussion?
> 
> Not since I've been actively following the list but I'll dig about in
> the archives later.
>  
>> Couldn't we just use a standard snapshot?  Take the latest nightly,
>> drop into some magical place in the tree, and then go into your module
>> and work from there?
> 
> Well, I was suggesting snapshots that might be:
> 
> 1) hdk (inc. jdk and jre)
> 2) jdk (inc. jre)
> 3) jre
> 
> but I think (?) you are suggesting using the one I overlooked:
> 
> 0) everything in classlib/trunk
> 
> I think 0) is going to get pretty big (but we should still create it)
> and I think we should actively support using 1) for development too.
> 
> Wasn't someone recently pleading with Eclipse to make smaller artifacts?
> ;-)

That's a totally different problem.

In the end, I don't care if we snapshot things like this to make it 
easier, but I'm really worried about what this will become.

Also, I think it would be better to be clear about the issues in here.

So assuming I understand the issues, I'm not against this as long as the 
world is indistinguishable if I do a svn co and a "make the world" 
top-level build, or just checkout a module and drop in a hdk above it in 
the tree.

AS a matter of fact, I think that the hdk is simple a tar of the junk 
created by a "make the world" build...


Also, for you, with multiple workspaces, I would imagine that your life 
would be better with this being resolvable via a "pointer" in the build 
properties (which defaults to "." or -ish), so you can have both a full 
tree around, as well as one or more hdk snapshots.

The following is an example of having the full tree ('full_checkout') at 
the same time as a hdk ('binary snapshot aka hdk') with three work areas 
(just the prefs checked out from SVN, the RMI contribution from Intel 
and the RMI contribution from ITC)

/your_see_drive :)
    /full_checkout/
           /deploy/
                artifacts
           / whatever/
    /binary snapshot aka hdk/
           /deploy/
                 artifacts
    /checkout_of_pref_only/
          build.props (points to /binary snapshot aka hdk/)
          /modules/
              /pref/
    /contribution_1_from_SVN/
          build.props (points to /full checkout/)
           /modules/
                /RMI from Intel/
    /contribution_2_from_SVN/
          build.props (points to /full checkout/)
           /modules/
                 /RMI_from_ITC/


That way, you can dork w/ the full_checkout, fix something, and then 
your other work environments that are pointing at it get that fix w/o 
any work.

I hope my crude art makes sense.

geir

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message