Return-Path: X-Original-To: apmail-felix-dev-archive@www.apache.org Delivered-To: apmail-felix-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 053086EA6 for ; Fri, 29 Jul 2011 08:00:29 +0000 (UTC) Received: (qmail 74167 invoked by uid 500); 29 Jul 2011 08:00:28 -0000 Delivered-To: apmail-felix-dev-archive@felix.apache.org Received: (qmail 73775 invoked by uid 500); 29 Jul 2011 08:00:21 -0000 Mailing-List: contact dev-help@felix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@felix.apache.org Delivered-To: mailing list dev@felix.apache.org Received: (qmail 73753 invoked by uid 99); 29 Jul 2011 08:00:18 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Jul 2011 08:00:18 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [74.125.82.177] (HELO mail-wy0-f177.google.com) (74.125.82.177) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Jul 2011 08:00:13 +0000 Received: by wyf23 with SMTP id 23so490366wyf.22 for ; Fri, 29 Jul 2011 00:59:51 -0700 (PDT) Received: by 10.227.55.129 with SMTP id u1mr1324163wbg.85.1311926391122; Fri, 29 Jul 2011 00:59:51 -0700 (PDT) Received: from [192.168.0.2] (AMontpellier-158-1-140-41.w90-42.abo.wanadoo.fr [90.42.203.41]) by mx.google.com with ESMTPS id fx12sm1523122wbb.25.2011.07.29.00.59.48 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 29 Jul 2011 00:59:49 -0700 (PDT) Subject: Re: Feedback Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Peter Kriens In-Reply-To: Date: Fri, 29 Jul 2011 09:59:46 +0200 Cc: dev@felix.apache.org Content-Transfer-Encoding: quoted-printable Message-Id: <60C40335-DB65-48D7-81EA-76259A2A6A36@aqute.biz> References: <2F9B375F-B5FD-4499-9039-02A9F9E82C89@aqute.biz> <51CD4693-BCF5-4BBB-AFCB-5F8A2CF0E24D@aqute.biz> <9F45351B-0DA5-41F3-9DB7-4A11DD72D2E1@aqute.biz> To: Tiger Gui X-Mailer: Apple Mail (2.1084) Rule #1, as long as I see you're engaged you do not have to worry about = the evaluation. So do not do things for my sake, do them because you = believe it is the best way forward. I think Tinkerpop and JUNG are a = great way to visualize, but do not take my word. I have the greatest = respect for people who do not listen to me because they are convinced = they know better. That out of the way. This whole things is very much about understanding and our visual brains = is by far the most powerful tool on earth to understand things. And = you're not taking advantage of it :-) We are looking for structure here, = the names are only relevant later. So to see structure, tables are = perfect. Which means you have to have short names. Just using the last = part of a package (com.springframework.jca, take jca). I.e. a name like = round1MergeBundle8 is ok to call b8. You can always later provide a list = of translations. This is also the reason why we need to use packages, = classes are too many off. Visualization is everything in this phase = because we need to find the rules of the structure. So lets retry. First give me a list of all groups (named g1 .. gn) and = their package content Group Contains Uses Used By = External g3 com.springframework.{jca,jdbc,...}, g4, g5, g7 g2 = org.w3.xml, javax.persistence, ... = com.oracle.g11.{impl,whatever} It might be easiest to generate HTML so you can use tables. Don't make = it look fancy, just use the tools to get insight in the structure: =20
Group Contains Uses Used By External
g3 com.springframework.{jca,jdbc,...},... g4, g5, g7 g2 org.w3.xml, = javax.persistence,com.oracle.g11.{impl,whatever}
Shortening the names is everything or else your eyes drown in a sea of = characters. Kind regards, Peter Kriens On 29 jul 2011, at 05:16, Tiger Gui wrote: > Hi Peter, >=20 > Thank you for your reply first, please see the following comments >=20 > 2011/7/28 Peter Kriens : >> It is good that you use a real project now. What I like to see how = many "bundles" we have after the first round, where you group strongly = connected packages. This should already simplify because the nr of = entities will be smaller. >>=20 >> I need to see a real case. Step 1 is ok, but maybe you can add = merging of bundles that have identical usesExternal (imports). >=20 > Yes, I do merge the bundles that have identical usesExternal > (imports), i described in the former mail, >=20 > We define sameUE: it menas the number of two bundles have same > usedExternal external package rely elements. >=20 > boolean condition2 =3D 3 * sameUE >=3D one.usedExternalList.size() + > two.usedExternalList.size(); >=20 > if(condition2 =3D=3D true) i will merge bundles one and two,In the = process > of real case test(Spring and Tomcat), i found that it's hardly to find > two bundles which have exactly the same usesExternal items, we should > also merge bundles who have *proportional* usesExternal items. In my > test case, it works good. >=20 >> And make sure you ignore all java.* to make the info smaller (in = OSGi, java is always provided by the VM). Can you just print out = something like >=20 > Yes, i have considered this situation, in the whole merge process, we > do not consider jdk supplies java.* classes. >=20 >>=20 >> name usesInternal usesExternal >> g1 g2 a,b,c >> g2 d,e,f >>=20 >=20 > Let's take example of Spring, this report is like this: > Name --- = usesInternal =09 > org.springframework.jms2 > = ---round1MergeBundle8,mergedBundle9,org.springframework.jca22,org.springfr= amework.scheduling15,org.springframework.context14,round1MergeBundle5 > org.springframework.jdbc11 --- = round1MergeBundle8,mergedBundle9=09 > org.springframework.orm13 > --- = round1MergeBundle8,org.springframework.jdbc11,mergedBundle9,round1MergeBun= dle3 > org.springframework.context14 > --- round1MergeBundle8,round1MergeBundle1,mergedBundle9 > org.springframework.scheduling15 > --- = round1MergeBundle8,mergedBundle9,org.springframework.context14,org.springf= ramework.jdbc11 > org.springframework.jca22 > --- = round1MergeBundle8,mergedBundle9,org.springframework.context14,org.springf= ramework.scheduling15 > round1MergeBundle1 > --- = round1MergeBundle8,mergedBundle9,org.springframework.metadata7 > round1MergeBundle3 > --- = round1MergeBundle8,org.springframework.context14,round1MergeBundle5 > round1MergeBundle4 --- = round1MergeBundle8,org.springframework.context14=09 > round1MergeBundle5 > --- = round1MergeBundle8,round1MergeBundle3,org.springframework.context14,merged= Bundle9 > mergedBundle9 > --- = round1MergeBundle8,round1MergeBundle5,org.springframework.context14,org.sp= ringframework.metadata7 > org.springframework.metadata7 --- none =09 > round1MergeBundle8 --- none >=20 > As usesExternal element list is too long, i abridged them, you can get > the report details with usesExternal elements in attach file > "bundles_relation.txt", and get the bundles details in attach file > "SpringSplitTest.analyse" >=20 >=20 >> I hope we see one big bundle which is the core and then have to find = rules to classify the remaining bundles. I expect there are the = following categories: >>=20 >> core implementation classes, lots of strongly connected = packages >> api api classes, do not refer to core, very few imports >> bridge refer strongly to core and have expensive imports >>=20 >>=20 >> At this stage, the trick is to do some work by hand until you find = you really understand the problem. >>=20 >> It would be perfect if you could take a look at Tinkerpop and JUNG. I = think it would be quite easy to visualize the graph of dependencies. >=20 > You mean my next step is developing a Tinkerpop or JUNG graphical view > for this report for user to adjust the bundles details manully ? Am i > right ? I will start learn Jung and start this job soon >=20 >>=20 >> Kind regards, >>=20 >> Peter Kriens >>=20 >>=20 >>=20 >>=20 >> On 26 jul 2011, at 16:24, Tiger Gui wrote: >>=20 >>> Hi Peter, >>>=20 >>> This is the whole application split algorithm here. After = application >>> source code analyse algorithm described here[1], we can know each >>> package/class use which packages/classes and be used by which >>> packages/classes. Now, we just discuss package here, we treat = package >>> as a single atom, each package has three important attributes, = usedBy, >>> usesExternal and usesInternal, just like below: >>>=20 >>> >> = sources=3D"/E:/GSoC/gsoc2011osgi/runtime-New_configuration/TomcatJava/bin"= >>> size=3D"30" usedBy=3D"8" usesInternal=3D"11" usesExternal=3D"14" = layer=3D"6" >>> cycle=3D"org.apache.catalina et al."> >>> >>> >>> = >>> >>> >>> = >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>>=20 >>> usedBy means who use current package; >>> usesInternal means which packages current package use in current >>> application source code; >>> usesExternal means which packages current pakcage use not in current >>> application's source code. >>>=20 >>> Now, we start split the whole application in to bundles, according = to >>> above algorithm source code analyse algorithm, we can also know >>> package cycles in current application. >>>=20 >>> 1. Treat each package cycle as a single bundle; >>> 2. Treat each packages not in cycle as a single bundle; >>> 3. Then we should decide which bundles can be merged into one new = bundle; >>>=20 >>> First round merge job: >>> 4. If one bundle's all usesInternal elements are in the other = bundle, >>> these two bundles should be merged into a new bundle; >>>=20 >>> Think about how to merge used by only bundle (bundle be used only by >>> other bundle, it does not rely on any other bundle): >>> Define two variable for used by only bundle: >>> sameUB: it means the number of two bundles have same usedBy = elements. >>> sameUE: it menas the number of two bundles have same usedExternal >>> external package rely elements. >>>=20 >>> boolean condition1 =3D 4 * sameUB >=3D one.usedByList.size() + >>> two.usedByList.size(); >>> boolean condition2 =3D 2 * sameUE >=3D one.usedExternalList.size() + >>> two.usedExternalList.size(); >>>=20 >>> if condition1 or condition2 is true, we should merge these two = usedBy >>> only bundle. >>>=20 >>> Merge the other bundles: >>> 5.The core problem is how to decide two bundles(bundle one and = bundle >>> two) can be merged or not. >>>=20 >>> Define 5 variables: >>> uiNumber: the sum of bundle one's usesInternal elements in bundle = two >>> number and bundle two's usesInternal elements in bundle one number; >>>=20 >>> ubNumber:the sum of bundle one's usedBy elements in bundle two = number >>> and bundle two's usedBy elements in bundle one number; >>>=20 >>> sameUI: the same usesInternal number bundle one and two have >>>=20 >>> sameUB: Be similar with used by only bundle's this variable >>>=20 >>> sameUE: Be similar with used by only bundle's this variable >>>=20 >>> Define these conditions: >>> boolean condition1 =3D 2 * uiNumber >=3D one.usesInternalList.size() = + >>> two.usesInternalList.size(); >>> boolean condition2 =3D 2 * ubNumber >=3D one.usedByList.size() + >>> two.usedByList.size() ; >>> boolean condition3 =3D 3.5 * sameUI >=3D one.usesInternalList.size() = + >>> two.usesInternalList.size(); >>> boolean condition4 =3D 4 * sameUB >=3D one.usedByList.size() + >>> two.usedByList.size(); >>> boolean condition5 =3D 3 * sameUE >=3D one.usedExternalList.size() + >>> two.usedExternalList.size(); >>>=20 >>> If any above condition is true, these two bundles can be merged. But >>> these are another problem, if bundle A can be merged with B, but it >>> also can be merged with C, now, we should decide merge A with B or A >>> with C. >>>=20 >>> Define the follow attribute: >>> int mergeFactor=3D 2 * (uiNumber + ubNumber) + 0.4 * (sameUI + = sameUB) + >>> 0.2 * sameUE - number; >>>=20 >>> If A and B's mergeFactor is 20 and A and C's mergeFactor is 30, we >>> should merge A and C. >>>=20 >>> This is current merge algorithm in OSGiMaker, i will keep on = improving >>> it, use class relationship factors or etc. You can find the source >>> code detail of this algorithm in class AnalyseJob of our project. >>>=20 >>> The attach file is the analyse result document OSGiMaker analyse >>> Tomcat's source code and split it into bundles, you can have a = review. >>>=20 >>>=20 >>> In fact, i did not want to bother you too much, but it seems that = you >>> have enough time to help me to improve it, this is a good thing. If >>> you have any advises, please let me know, let's improving it = together >>> :-) >>>=20 >>> Thank you >>>=20 >>> [1] = http://code.google.com/p/osgimaker/wiki/Implement_of_project_analyse_algor= ithm >>>=20 >>> 2011/7/26 Peter Kriens : >>>> Well, I do not think I am eager but I have a hard time getting a = feeling where you are. You do not have to send reports, I want to see = intermediate results and discuss issues if there are any. As I said = earlier, it is not clear to me yet what the best algorithm is so that = need to be worked out before we do the gui stuff. >>>>=20 >>>> Kind regards, >>>>=20 >>>> Peter Kriens >>>>=20 >>>>=20 >>>>=20 >>>> On 26 jul 2011, at 11:24, Tiger Gui wrote: >>>>=20 >>>>> In my schedule, i will report current status to you tomorrow as i >>>>> think i can get a usable version today, the whole analyse and = split >>>>> algorithm is complex i have to organize a document to describe it >>>>> clearly. As you are really eager to see its progress, it is OK, i = will >>>>> start the report now >>>>>=20 >>>>> 2011/7/26 Peter Kriens : >>>>>> If that is the case you have to provide more on going feedback. = What happened to the analysis by hand? >>>>>>=20 >>>>>> Kind regards, >>>>>>=20 >>>>>> Peter Kriens >>>>>>=20 >>>>>>=20 >>>>>> On 26 jul 2011, at 11:02, Tiger Gui wrote: >>>>>>=20 >>>>>>> No, Peter, i am really working hard for this project, you can = check >>>>>>> the progress here[1]. Now, this tool can analyse a whole project = and >>>>>>> export the its analyse result to bundles( I test it in Spring = and >>>>>>> Tomcat project), if possible, you can install it in your Eclipse = and >>>>>>> have a trial of it. But, i am still improving the split = algorithm as >>>>>>> the current algorithm is not working perfect. >>>>>>>=20 >>>>>>> I will supply a document about the current status of this = project and >>>>>>> a simple guide for you to have a trial of it. I am really = working very >>>>>>> hard for it these days :-( >>>>>>>=20 >>>>>>> [1] http://code.google.com/p/osgimaker/updates/list >>>>>>>=20 >>>>>>> 2011/7/26 Peter Kriens : >>>>>>>> I am getting the feeling that you're not working very hard on = this project and only does something just for the evaluations ... >>>>>>>>=20 >>>>>>>> Peter Kriens >>>>>>>>=20 >>>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>>=20 >>>>>>> -- >>>>>>> Best Regards >>>>>>> ---------------------------------------------------- >>>>>>> Tiger Gui [tigergui1990@gmail.com] >>>>>>=20 >>>>>>=20 >>>>>=20 >>>>>=20 >>>>>=20 >>>>> -- >>>>> Best Regards >>>>> ---------------------------------------------------- >>>>> Tiger Gui [tigergui1990@gmail.com] >>>>=20 >>>>=20 >>>=20 >>>=20 >>>=20 >>> -- >>> Best Regards >>> ---------------------------------------------------- >>> Tiger Gui [tigergui1990@gmail.com] >>> >>=20 >>=20 >=20 >=20 >=20 > --=20 > Best Regards > ---------------------------------------------------- > Tiger Gui [tigergui1990@gmail.com] >