harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oliver Deakin <oliver.dea...@googlemail.com>
Subject Re: Supporting working on a single module?
Date Fri, 12 May 2006 10:44:05 GMT
Geir Magnusson Jr wrote:
> Oliver Deakin wrote:
>> Geir Magnusson Jr wrote:
>>> Mark Hindess wrote:
>>>> On 9 May 2006 at 10:32, Geir Magnusson Jr <geir@pobox.com> wrote:
>>>>> Mark Hindess wrote:
>>>>>> As the Harmony Classlib grows, I think that being able to work on
>>>>>> single module (or some subset of the modules) will become the
>>>>>> typical way of working for many (perhaps even most) contributors.
>>>>> Sure, that makes sense.
>>>>>> So I think we need a plan to support this.  I also think that
>>>>>> forcing ourselves to support this mode of working sooner rather than
>>>>>> later will help us keep from accidentally breaking the modularity
>>>>>> the current build/development process.
>>>>> Can you not work on a single module now?
>>>> Yes.  But only by checking out everything.
>>> Wouldn't you do that anyway?
>>> I guess thats the mystery to me.  I figured that someone would do 
>>> the full checkout, and go from there.
>> I think it's good if we can cater not only for those developers who 
>> want to completely checkout classlib trunk, but also for those who 
>> want to work only on a single module. When I say "work on a single 
>> module", I mean only checking that module's directory out of SVN, and 
>> not the code for all the other modules, thereby saving download time 
>> and disk space, not to mention the subsequent reduced build times of 
>> only recompiling a single module.
> Eh. :)  We have agreement in a root issue, but not as you describe. 
> IOW, I agree that it should be that after a full build, you should be 
> able to dive into a module and work and re-build in there in isolation.
> However, I have little sympathy for the reduced download time or 
> diskspace argument.
> I *do* sympathize with [psychos like] Mark who has 4 builds at once, 
> and see that it would be efficient to have multiple independent 'dev 
> subtrees' pointing at one or more pre-built trees.

It seems we have agreement that being able to rebuild a single module in 
isolation is a good idea - we have different ideas about how that may 
then be used, but I think both are valid and possible once the proper 
modularisation of code and build scripts exists. I guess once the build 
system is in place, it's up to the developer how they wish to use it :)

>> Currently a developer cannot work on both the native and Java code 
>> for a module without checking out the whole of classlib trunk, 
>> because the native source directories and Ant build script associated 
>> with them are completely separate from the modules. IMHO it would be 
>> great if we could get the build scripts and code repository layout 
>> into a state where a developer can just checkout a module, grab an 
>> hdk from some location and start changing and rebuilding both Java 
>> and native code (and also running tests against the rebuilt libraries).
> This is mixing two issues.
> yes, I think we should put the native code into each module, so you 
> can just work in one module, both native and Java.
> Yes, I see a reason why you would want to have a pre-build snapshot to 
> use for multiple dev subtrees, for "known good" regression testing, etc.
> But I have trouble with linking them.

ok, fair enough - I have mixed these two issues together. I think this 
stems from what I said above - we just have different ideas about how 
the modularisation will be used once it's in place. When I think of 
modularisation, I imagine a developer grabbing the source just for the 
module they are interested in and a snapshot build (hdk) and working 
away with it - thus I see the modularisation and the creation of an hdk 
as being linked. As you say, really they are two separate issues.

>> To do this there are at least three steps needed, as far as I can see:
>> 1) Refactor the native code into the modular structure we currently 
>> have for Java code and tests. This has been spoken about before at 
>> [1]. The native code would be located within each module at 
>> modules/<module name>/src/main/native. As a starting point, can I 
>> propose that the natives be broken down in the following way:
>> modules/archive/src/main/native/
>>                                        |-------archive/
>>                                        |-------zip/
>> +------zlib/                          modules/auth/src/main/native/
>>                                        +-------auth/
>> modules/luni/src/main/native/
>>                                        |--------common/
>>                                        |--------fdlibm/
>>                                        |--------launcher/
>>                                        |--------luni/
>>                                        |--------pool/
>>                                        |--------port/
>>                                        |--------sig/
>>                                        |--------thread/
>>                                        |--------vmi/
>>                                        +-------vmls/
>> modules/prefs/src/main/native/
>>                                       +-------prefs/
>> modules/text/src/main/native/
>>                                        |-------text/
>>                                        +------unicode/ (comes from 
>> the icu4c zips stored currently in depends)
> W/o thinking too hard about it, this works for me just fine.

Great - I am starting to look at how shared includes can be handled 
across modules (as Mark alluded to in his earlier post in this thread 
[1]), and at what changes will be required to split the natives into 
these locations. I will be taking this in small steps, trying to get the 
foundation and "easy" parts done first, and raising a JIRA for each step 
rather than in one monolithic change.

>> Additionally, each module may have an include directory that contains 
>> header files required across its native components. Each native 
>> component would contain subdirectories for its shared and platform 
>> specific code.
> Ok, as far as I undertsand it.
>> 2) Alter the global build scripts to start creating an hdk under the 
>> deploy directory, which contains a jdk which contains a jre. As Mark 
>> described before, this hdk would probably look something like:
>> deploy/hdk/
>>               |--------jdk/
>>               |              |-------jre/    (already being created 
>> at the moment as deploy/jre)
>>               |              +------include/    (already being 
>> created at the moment as deploy/include)
>>               |--------include/
>>               +-------lib/
>> The hdk/include directory would be much like the current 
>> native-src/<platform>/include directories, and would contain a 
>> minimal set of header files that define an interface between native 
>> modules. It would be the responsibility of the individual module to 
>> copy its required headers to this directory as a prerequisite for 
>> building.
>> The hdk/lib directory would exist on Windows platforms, and would 
>> contain .lib files for each native component.
>> An hdk snapshot would then be a zip of the hdk directory, and should 
>> provide everything necessary for a developer to build both native and 
>> Java code against.
> Mmmmm.  I'm struggling w/ having the /jdk part of it.

What is it you don't like about /jdk?

>> 3) Modularise native and Java build scripts so that the individual 
>> module's Ant scripts are capable of building native and Java source 
>> against an hdk, rather than having its code compiled as part of a 
>> global build.
> 1) Re modularize build scripts, Yes!  I thought it was this way 
> already but that it wasn't used, because that's how we added security2 
> to the build - by just invoking from the master.  IMO, that's how it 
> all should have been done.
> 2) re against an HDK, fine, but never assume it's location.  Make sure 
> that I can put it anywhere on a filesystem and it works.  That way you 
> can have multiple dev trees pointing to it, or let me easily "A/B" 
> test by just changing a property somewhere.
>> The rebuilt libraries would be placed back into the hdk directory, so 
>> that the developers local hdk always contains their latest built code 
>> (much as the current deploy/jre does).
> I think this is a bad idea.
> First, I think we want to support an HDK being used by multiple dev 
> trees, so you don't want those trees dropping new untested into it.
> Second, think about it - do you really want to be modifying a 
> known-good snapshot with code you are debugging?  How do you "reset"?
> I'd keep the HDK separate from "deploy".
> Given each module knows it's artifacts, it should be able to 
> dynamically construct the CPs and library paths as it needs to for 
> in-module testing, which is the point of this exercise...?
> geir
Does it matter if we overlay newly built libraries onto a snapshot? 
Isn't "resetting" as simple as unzipping the snapshot archive again from 
whereever you have it stored locally?

The way I see it is:
 - If you're working on an entire checkout of classlib/trunk, you don't 
really need a snapshot build, because you can rebuild the deploy 
directory from scratch. How do you "reset" in this case? If you really 
don't want the changes you've made, you can always SVN revert and rebuild ;)
 - If you're working on a single module and don't have the whole of 
classlib checked out, you need to build against a snapshot. In this case 
I think you'd want to be able to overlay your new libraries onto the 
snapshot - otherwise what other option do you have to get a complete 
build without checking out the rest of trunk?

IMHO it's not a problem to overwrite the libraries with new altered ones 
- it's what we do now with the deploy directory (whether it's a snapshot 
or built from scratch).


>> Additionally, I agree with Nathan [2] that allowing a developer 
>> working in a modular way to run the entire test suite against their 
>> work-in-progress code would be a good thing, and this could be done 
>> by bundling tests and their resources in with an hdk.
>> Regards,
>> Oliver
>> [1] 
>> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200602.mbox/%3cc3755b3a0602100609x1b79da6q@mail.gmail.com%3e

>> [2] 
>> http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200605.mbox/%3c000f01c673e2$92e74130$0e01a8c0@LITTLEGUY%3e

> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org

Oliver Deakin
IBM United Kingdom Limited

Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org

View raw message