Mailing-List: contact dev-help@geronimo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@geronimo.apache.org
Received-SPF: neutral (asf.osuosl.org: local policy)
Mime-Version: 1.0 (Apple Message framework v752.2)
In-Reply-To: <6C4543AF-375A-4AA7-9516-D5805BD5909C@yahoo.com>
References: <20060817185115.63202.qmail@web31709.mail.mud.yahoo.com>
 <FC18717D-5DF2-455D-A008-D8107D4F07DA@planet57.com>
 <15BA70B0-136A-44B9-926F-AF7738315362@apache.org>
 <6C4543AF-375A-4AA7-9516-D5805BD5909C@yahoo.com>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <C7915DC2-012E-403A-B6C4-59BE7EF86C6C@apache.org>
Content-Transfer-Encoding: 7bit
From: Bill Dudney <bdudney@apache.org>
Subject: Re: m2 build - validating
Date: Mon, 21 Aug 2006 07:07:09 -0600
To: dev@geronimo.apache.org

Hi David,

On Aug 21, 2006, at 1:47 AM, David Jencks wrote:

>
> On Aug 20, 2006, at 3:36 PM, Bill Dudney wrote:
>
>> Hi All,
>>
>> Sorry its taken me so long to get this posted. I had a bunch of  
>> travel related headaches thur and fri of last week...
>>
>> I've adopted http://issues.apache.org/jira/browse/GERONIMO-2326 as  
>> the JIRA to track the first part of the problem (the datasource  
>> deployment stuff) with the current trunk and deployment.
>>
>> I've just attached a patch to the issue that makes a small change  
>> to the CAR plugin to include RAR's in the dependencies that are  
>> copied into the assembly repository. I am fairly sure this needs  
>> to be done as the RAR's end up in the repository on 1.1 and the  
>> DatabaseInfo class requires them to be there (albeit with a  
>> comment that says EVIL!!! :). However there might be something  
>> going on there I'm not aware of.
>
> This is basically a wrong thing to do.  A dependency on the rar  
> file doesn't do anything except require it to be there, which is  
> bogus since in fact once the car file is generated it doesn't have  
> to be there.  The tranql rars (and the activemq rar) are included  
> only for the convenience of users wishing to deploy additional  
> datasources without downloading the rar file themselves.
>

Yes but the Datasource deployment code in trunk depends on the RAR's  
being in the repository, after spending some time digging around the  
1.1.1 code there has been a lot of change in the way this stuff  
works. Would be great to get trunk and 1.1.1 more in line as it might  
fix many of the problems.

> There's been discussion for a really long time about not copying  
> the contents of j2ee artifacts into the car file but instead  
> including them in the classloader in some other way.  This sounds  
> nice but is fraught with difficulties such as....
> -how do you make a classloader that reads the jars in a war lib dir  
> inside an ear  (nested jar classloading)
> -or, how do you figure out to unpack the j2ee artifacts in the repo
> - how do you set up a variety of classloaders that only load some  
> stuff from an ear: e.g. right now an ear classloader doesn't see  
> anything in any web apps.
>
> these are all soluble but will take a bunch of work to decide what  
> to do and implement.

i'm game to help with the work lets get started.

>
>>
>> One quick note on this change. The RAR's do end up copied into the  
>> CAR (without this change) but, the DatabaseInfo class is expecting  
>> (at least) the Derby related RAR's to be directly in the repo.  
>> Take a look here http://tinyurl.com/jynnh. the list of *RAR_NAME  
>> fields are all used to load the RAR's directly from the repository  
>> (http://tinyurl.com/kyhve getRAR static method on line 1036). As  
>> the comments says this is rather hard coded. I've not had time to  
>> get brain cells focused on the solution. Seems that we should be  
>> able to do some kind of maven magic though to find the tranql/ 
>> tranql-connector/rar instead of hardcoding the path. But again I  
>> don't know the history so perhaps there is a reason for the EVIL :)
>
> I think that the code for 1.1 is greatly improved over what's in  
> trunk, and I hope aaron ports it soon.  Again, using rar's as  
> dependencies is a bad idea.  I think something got left out of the  
> m2 build, in the m1 build there was a way to specify that some jars  
> just get put in the repo even though they aren't dependencies of  
> any car.  I don't know if this is possible in m2, but the tranql  
> rars are not getting installed at least on my machine.
>

Right the patch to the m2-car-plugin which is in the patch I attached  
to 2326 fixes this. Essentially the plugin was ignoring RAR files.  
With the patch it includes RAR's in the repository.

>>
>> So the patch attached to 2326 gets us almost there, but there is  
>> still the issue with the IncompatiblClassChangeError coming up. I  
>> posted a partial stack trace in the issue. Any pointers, thoughts  
>> or other things that could help me track that one down are most  
>> welcome. I have to bail for a couple of hours but will be back on  
>> line around 7:00MST for a bit.
>>
>> Also David J, if you could take a look at the pom changes and make  
>> sure they are in line with your recommendations. I could not find  
>> another car to have system-database depended on that would bring  
>> in the dependencies and system-database looked to be the most  
>> logical place to add add a jar dependency. I added the jar's to  
>> the gbean-deployer/pom.xml because the CNFExceptions were coming  
>> from classes in this package and again no obvious CAR to add to  
>> the mix.
>
> Unfortunately I don't like almost all your proposed pom changes.
>

No problem, that is why I asked you to look as I figured you'd have  
an opinion :-)

> We do need to add the geronimo-test-ddbean jar to some module, but  
> I think it should be added to j2ee-deployer since jsr-88 is pretty  
> irrelevant to gbean deployment.

Good point, I will move the test-ddbean depndency to the j2ee- 
deployer I think the geronimo-j2ee-deployment_1.1_spec should also  
move to the j2ee-deployer. I am fairly confident though that the  
changes to system-database/pom.xml should remain where they are. Do  
you agree?

> With just this change I'm getting an NPE rather than a CNFE (which  
> I got with unchanged trunk g).
>

The NPE is coming from the missing RAR file. You have to rebuild the  
CAR plugin and then rebuild all the configs then reassemble. Should  
have put that in the JIRA (and have now).

> I have yet to reproduce the IncompatibleClassChangeError but I  
> don't see any way it can be caused by anything other than using 2  
> different versions of a jar somehow.  Perhaps one of the jars in  
> lib is a different version than what is in the g. repo???  If you  
> can figure out what class it is that might be helpful.
>

I figured the same thing so I did a shallow bootstrap thinking that  
would get rid of everything and make this go away. I get the same  
error. I spent some time trying to find the class that its failing on  
and its not that obvious at least it is confusing the Eclipse debugger.

I just finished a fresh update and bootstrap, trying again and will  
let you know what I find.

> I'm to tired to work on this more tonight, I'll try to look more in  
> the morning.
>

thanks,

-bd

> thanks
> david jencks
>
>>
>> Thanks!
>>
>> bd-
>>
>> On Aug 17, 2006, at 3:07 PM, Jason Dillon wrote:
>>
>>> And it seems I have the magic incantation now :-)  Going to see  
>>> how well I can cast it on the car plugin later today.
>>>
>>> --jason
>>>
>>>
>>> On Aug 17, 2006, at 11:51 AM, anita kulshreshtha wrote:
>>>
>>>>
>>>>
>>>> --- Jason Dillon <jason@planet57.com> wrote:
>>>>
>>>>> IMO we should not overload the <scope> mechanism here, but  
>>>>> define new
>>>>>
>>>>> configuration elements that allow us to add the additional  
>>>>> metadata
>>>>> needed....
>>>>
>>>>     Bill, if you do not want to improve the existing car-maven- 
>>>> plugin,
>>>> you will need to use the scope.
>>>>
>>>>  just like how I added classpathPrefix to be used to
>>>>> control the prefix for manifest entries.
>>>>
>>>> IMO, the classpath should be constructed dynamically using the
>>>> transitive dependencies of the modules, i.e for j2ee-system  
>>>> config use
>>>> dependencies of the system module. The way it is now is very  
>>>> hard to
>>>> maintain. Every time a dependency is changed in the system  
>>>> module, most
>>>> executable configurations must be updated manually. The current  
>>>> way of
>>>> maintaining hard coded dependencies in the classPath element are  
>>>> only
>>>> marginally better than maintaining a large string of the original
>>>> implementaion. Adding prefix to these dependencies is trivial.
>>>>
>>>> Thanks
>>>> Anita
>>>>
>>>>
>>>>>
>>>>> Then the m2 dependencies would only really be used to control  
>>>>> build
>>>>> order for cars.
>>>>>
>>>>> --jason
>>>>>
>>>>>
>>>>> On Aug 17, 2006, at 8:59 AM, anita kulshreshtha wrote:
>>>>>
>>>>>> Oops.., the correct link is :
>>>>>>
>>>>>> http://mail-archives.apache.org/mod_mbox/geronimo-dev/ 
>>>>>> 200605.mbox/%
>>>>>
>>>>>> 3c20060523131245.44642.qmail@web31713.mail.mud.yahoo.com%3e
>>>>>>
>>>>>> Thanks
>>>>>> Anita
>>>>>>
>>>>>> --- anita kulshreshtha <a_kulshre@yahoo.com> wrote:
>>>>>>
>>>>>>> inline...
>>>>>>>
>>>>>>> --- David Jencks <david_jencks@yahoo.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> On Aug 16, 2006, at 4:33 PM, Bill Dudney wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> i've been using the m2 build for several days now and I've
>>>>>>> noticed
>>>>>>>>
>>>>>>>>> that while it works well there are several details that are
>>>>> still
>>>>>>>
>>>>>>>>> not nailed down. Particularly I've been hitting lots of
>>>>>>> dependency
>>>>>>>>
>>>>>>>>> issues around deployment. So what I've started doing is  
>>>>>>>>> slogging
>>>>>>>
>>>>>>>>> through each of them one at a time, posting a jira and a  
>>>>>>>>> patch.
>>>>>>>>>
>>>>>>>>> It struck me that there are probably similar issues throughout
>>>>>>> the
>>>>>>>>
>>>>>>>>> server WRT the m2 build.
>>>>>>>>>
>>>>>>>>> I'm open to other methods (and would love to hear of a silver
>>>>>>>>> bullet:) but seems to me that we need to basically hit
>>>>> everything
>>>>>>>
>>>>>>>>> in the console and tools and such and make sure it works so we
>>>>>>> can
>>>>>>>>
>>>>>>>>> be sure the dependencies are correct. While I don't think I'll
>>>>> be
>>>>>>>
>>>>>>>>> able to hit 'everything' I'll try to poke on most of the  
>>>>>>>>> console
>>>>>>>
>>>>>>>>> and the CLI tools and make sure that it 'works'.
>>>>>>>>>
>>>>>>>>> My plan of attack:
>>>>>>>>>
>>>>>>>>> 1 - provide patches for the stuff i know about now
>>>>>>> (tranql/tranql-
>>>>>>>>> connector is missing for example from the repository)
>>>>>>>>> 2 - finish getting deployment working from the console (data
>>>>>>>>> sources, ejb-jar's, wars etc)
>>>>>>>>> 3 - poke on the rest of the console
>>>>>>>>> 4 - deploy daytrader
>>>>>>>>> 5 - anything else anyone comes up with
>>>>>>>>>
>>>>>>>>> I will be posting bunches of jira's and fixes over the next  
>>>>>>>>> few
>>>>>>>>> days as I work through this stuff (unless someone has a better
>>>>>>> idea
>>>>>>>>
>>>>>>>>> about how to tackle it).
>>>>>>>>
>>>>>>>> This is great that you are taking a look at this.  Here are  
>>>>>>>> some
>>>>>>> tips
>>>>>>>>
>>>>>>>> that may help fix dependency problems:
>>>>>>>>
>>>>>>>> 1. The modules use resources2/META-INF/geronimo-dependency.xml
>>>>>>> files
>>>>>>>>
>>>>>>>> to specify transitive dependencies.  We could use a lot more of
>>>>>>>> these.  You may find the best solution to a classpath  
>>>>>>>> problem is
>>>>>>>> adding one of these.
>>>>>>>>
>>>>>>>> 2. There are often several ways to get a jar into a  
>>>>>>>> classloader,
>>>>>>> such
>>>>>>>> as
>>>>>>>>   a) depending on a car, from a car
>>>>>>>>   b) depending on a jar from a jar (using
>>>>> geornimo-dependency.xml)
>>>>>>>>   c) depending directly on the jar from a car
>>>>>>>> I'd say this is the order of preference
>>>>>>>>
>>>>>>>> 3) It is really important that the builder cars don't start any
>>>>>>>> runtime cars: if you violate this rule the packaging plugin is
>>>>>>> likely
>>>>>>>>
>>>>>>>> to stop working.  You can have 2.a and enforce this rule by  
>>>>>>>> using
>>>>>>> the
>>>>>>>>
>>>>>>>> <scope> element in the dependency and setting it to
>>>>>>> <scope>classes</
>>>>>>>> scope>.  This means the dependency's classloader will be
>>>>>>> constructed
>>>>>>>>
>>>>>>>> and available for use but that no services from the car will be
>>>>>>>> started.
>>>>>>>
>>>>>>>    This might be helpful in adding dependency using 2.a -
>>>>>>>
>>>>>>>
>>>>>> http://mail-archives.apache.org/mod_mbox/geronimo-dev/ 
>>>>>> 200605.mbox/
>>>>>> browser
>>>>>>>
>>>>>>> Thanks
>>>>>>> Anita
>>>>>>>>
>>>>>>>> This is in my experience somewhat nerve wracking and gruesome
>>>>> work,
>>>>>>>
>>>>>>>> so I really appreciate your taking it on, and I'll do  
>>>>>>>> whatever I
>>>>>>> can
>>>>>>>>
>>>>>>>> to help you with it.
>>>>>>>>
>>>>>>>> thanks
>>>>>>>> david jencks
>>>>>>>>
>>>>>>>>>
>>>>>>>>> TTFN,
>>>>>>>>>
>>>>>>>>> -bd-
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> __________________________________________________
>>>>>>> Do You Yahoo!?
>>>>>>> Tired of spam?  Yahoo! Mail has the best spam protection around
>>>>>>> http://mail.yahoo.com
>>>>>>>
>>>>>>
>>>>>>
>>>>>> __________________________________________________
>>>>>> Do You Yahoo!?
>>>>>> Tired of spam?  Yahoo! Mail has the best spam protection around
>>>>>> http://mail.yahoo.com
>>>>>
>>>>>
>>>>
>>>>
>>>> __________________________________________________
>>>> Do You Yahoo!?
>>>> Tired of spam?  Yahoo! Mail has the best spam protection around
>>>> http://mail.yahoo.com
>>>
>>
>