felix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Kriens <peter.kri...@aqute.biz>
Subject Re: Feedback
Date Fri, 29 Jul 2011 07:59:46 GMT
Rule #1, as long as I see you're engaged you do not have to worry about the evaluation. So
do not do things for my sake, do them because you believe it is the best way forward. I think
Tinkerpop and JUNG are a great way to visualize, but do not take my word. I have the greatest
respect for people who do not listen to me because they are convinced they know better. That
out of the way.

This whole things is very much about understanding and our visual brains is by far the most
powerful tool on earth to understand things. And you're not taking advantage of it :-) We
are looking for structure here, the names are only relevant later. So to see structure, tables
are perfect. Which means you have to have short names. Just using the last part of a package
(com.springframework.jca, take jca). I.e. a name like round1MergeBundle8 is ok to call b8.
You can always later provide a list of translations. This is also the reason why we need to
use packages, classes are too many off. Visualization is everything in this phase because
we need to find the rules of the structure.

So lets retry. First give me a list of all groups (named g1 .. gn) and their package content


Group  Contains                              Uses           Used By      External
g3     com.springframework.{jca,jdbc,...},   g4, g5, g7     g2           org.w3.xml, javax.persistence,
       ...                                                               com.oracle.g11.{impl,whatever}

It might be easiest to generate HTML so you can use tables. Don't make it look fancy, just
use the tools to get insight in the structure:
 
<html>
   <body>
       <table>
          <tr>
             <th>Group</th>
             <th>Contains</th>
             <th>Uses</th>
             <th>Used By</th>
             <th>External</th>
          </tr>
          <tr>
	    <td>g3</td>
	    <td>com.springframework.{jca,jdbc,...},...</td>
	    <td>g4, g5, g7</td>
	    <td>g2</td>
	    <td>org.w3.xml, javax.persistence,com.oracle.g11.{impl,whatever}</td>
          </tr>
        </table>
    </body>
</html>

Shortening the names is everything or else your eyes drown in a sea of characters.

Kind regards,

	Peter Kriens



On 29 jul 2011, at 05:16, Tiger Gui wrote:

> Hi Peter,
> 
> Thank you for your reply first, please see the following comments
> 
> 2011/7/28 Peter Kriens <peter.kriens@aqute.biz>:
>> It is good that you use a real project now. What I like to see how many "bundles"
we have after the first round, where you group strongly connected packages. This should already
simplify because the nr of entities will be smaller.
>> 
>> I need to see a real case. Step 1 is ok, but maybe you can add merging of bundles
that have identical usesExternal (imports).
> 
> Yes, I do merge the bundles that have identical usesExternal
> (imports), i described in the former mail,
> 
> We define sameUE: it menas the number of two bundles have same
> usedExternal external package rely elements.
> 
> boolean condition2 =  3 * sameUE >= one.usedExternalList.size() +
> two.usedExternalList.size();
> 
> if(condition2 == true) i will merge bundles one and two,In the process
> of real case test(Spring and Tomcat), i found that it's hardly to find
> two bundles which have exactly the same usesExternal items, we should
> also merge bundles who have  *proportional* usesExternal items. In my
> test case, it works good.
> 
>> And make sure you ignore all java.* to make the info smaller (in OSGi, java is always
provided by the VM). Can you just print out something like
> 
> Yes, i have considered this situation, in the whole merge process, we
> do not consider jdk supplies java.* classes.
> 
>> 
>> name            usesInternal            usesExternal
>> g1              g2                      a,b,c
>> g2                                      d,e,f
>> 
> 
> Let's take example of Spring, this report is like this:
> Name				---						usesInternal					
> org.springframework.jms2
> ---round1MergeBundle8,mergedBundle9,org.springframework.jca22,org.springframework.scheduling15,org.springframework.context14,round1MergeBundle5
> org.springframework.jdbc11     ---		round1MergeBundle8,mergedBundle9	
> org.springframework.orm13
> ---		round1MergeBundle8,org.springframework.jdbc11,mergedBundle9,round1MergeBundle3
> org.springframework.context14
> ---		round1MergeBundle8,round1MergeBundle1,mergedBundle9
> org.springframework.scheduling15
> ---		round1MergeBundle8,mergedBundle9,org.springframework.context14,org.springframework.jdbc11
> org.springframework.jca22
> ---		round1MergeBundle8,mergedBundle9,org.springframework.context14,org.springframework.scheduling15
> round1MergeBundle1
> ---		round1MergeBundle8,mergedBundle9,org.springframework.metadata7
> round1MergeBundle3
> ---		round1MergeBundle8,org.springframework.context14,round1MergeBundle5
> round1MergeBundle4     ---		round1MergeBundle8,org.springframework.context14	
> round1MergeBundle5
> ---		round1MergeBundle8,round1MergeBundle3,org.springframework.context14,mergedBundle9
> mergedBundle9
> ---		round1MergeBundle8,round1MergeBundle5,org.springframework.context14,org.springframework.metadata7
> org.springframework.metadata7     ---		none		
> round1MergeBundle8     ---			none
> 
> As usesExternal element list is too long, i abridged them, you can get
> the report details with usesExternal elements in attach file
> "bundles_relation.txt", and get the bundles details in attach file
> "SpringSplitTest.analyse"
> 
> 
>> I hope we see one big bundle which is the core and then have to find rules to classify
the remaining bundles. I expect there are the following categories:
>> 
>> core            implementation classes, lots of strongly connected packages
>> api             api classes, do not refer to core, very few imports
>> bridge          refer strongly to core and have expensive imports
>> 
>> 
>> At this stage, the trick is to do some work by hand until you find you really understand
the problem.
>> 
>> It would be perfect if you could take a look at Tinkerpop and JUNG. I think it would
be quite easy to visualize the graph of dependencies.
> 
> You mean my next step is developing a Tinkerpop or JUNG graphical view
> for this report for user to adjust the bundles details manully ? Am i
> right ? I will start learn Jung and start this job soon
> 
>> 
>> Kind regards,
>> 
>>        Peter Kriens
>> 
>> 
>> 
>> 
>> On 26 jul 2011, at 16:24, Tiger Gui wrote:
>> 
>>> Hi Peter,
>>> 
>>> This is the whole application split algorithm here. After application
>>> source code analyse algorithm described here[1], we can know each
>>> package/class use which packages/classes and be used by which
>>> packages/classes. Now, we just discuss package here, we treat package
>>> as a single atom, each package has three important attributes, usedBy,
>>> usesExternal and usesInternal, just like below:
>>> 
>>> <package name="org.apache.catalina.deploy"
>>> sources="/E:/GSoC/gsoc2011osgi/runtime-New_configuration/TomcatJava/bin"
>>> size="30" usedBy="8" usesInternal="11" usesExternal="14" layer="6"
>>> cycle="org.apache.catalina et al.">
>>>      <packageRef name="org.apache.catalina.core" type="usedBy"/>
>>>      <packageRef name="org.apache.catalina.startup" type="usedBy"/>
>>>      <packageRef name="org.apache.catalina.deploy" type="usedBy"/>
>>>      <packageRef name="org.apache.catalina.authenticator" type="usedBy"/>
>>>      <packageRef name="org.apache.catalina" type="usedBy"/>
>>>      <packageRef name="org.apache.catalina.mbeans" type="usedBy"/>
>>>      <packageRef name="org.apache.catalina.connector" type="usedBy"/>
>>>      <packageRef name="org.apache.catalina.realm" type="usedBy"/>
>>>      <packageRef name="java.lang" type="usesExternal"/>
>>>      <packageRef name="java.io" type="usesExternal"/>
>>>      <packageRef name="org.apache.catalina.deploy" type="usesInternal"/>
>>>      <packageRef name="org.apache.catalina.util" type="usesInternal"/>
>>>      <packageRef name="java.util" type="usesExternal"/>
>>>      <packageRef name="org.apache.juli.logging" type="usesInternal"/>
>>>      <packageRef name="org.apache.tomcat.util.res" type="usesInternal"/>
>>>      <packageRef name="java.beans" type="usesExternal"/>
>>>      <packageRef name="(default package)" type="usesExternal"/>
>>>      <packageRef name="org.apache.catalina" type="usesInternal"/>
>>>      <packageRef name="org.apache.catalina.mbeans" type="usesInternal"/>
>>>      <packageRef name="javax.management" type="usesExternal"/>
>>>      <packageRef name="namingResources" type="usesExternal"/>
>>>      <packageRef name="javax.naming" type="usesExternal"/>
>>>      <packageRef name="org.apache.naming" type="usesInternal"/>
>>>      <packageRef name="java.lang.reflect" type="usesExternal"/>
>>>      <packageRef name="javax.servlet" type="usesInternal"/>
>>>      <packageRef name="javax.servlet.annotation" type="usesInternal"/>
>>>      <packageRef name="org.apache.catalina.order" type="usesExternal"/>
>>>      <packageRef name="java.net" type="usesExternal"/>
>>>      <packageRef name="webXml" type="usesExternal"/>
>>>      <packageRef name="webXml.version" type="usesExternal"/>
>>>      <packageRef name="webxml" type="usesExternal"/>
>>>      <packageRef name="org.apache.catalina.core" type="usesInternal"/>
>>>      <packageRef name="javax.servlet.descriptor" type="usesInternal"/>
>>>    </package>
>>> 
>>> usedBy means who use current package;
>>> usesInternal means which packages current package use in current
>>> application source code;
>>> usesExternal means which packages current pakcage use not in current
>>> application's source code.
>>> 
>>> Now, we start split the whole application in to bundles, according to
>>> above algorithm source code analyse algorithm, we can also know
>>> package cycles in current application.
>>> 
>>> 1. Treat each package cycle as a single bundle;
>>> 2. Treat each packages not in cycle as a single bundle;
>>> 3. Then we should decide which bundles can be merged into one new bundle;
>>> 
>>> First round merge job:
>>> 4. If one bundle's all usesInternal elements are in the other bundle,
>>> these two bundles should be merged into a new bundle;
>>> 
>>> Think about how to merge used by only bundle (bundle be used only by
>>> other bundle, it does not rely on any other bundle):
>>> Define two variable for used by only bundle:
>>> sameUB: it means the number of two bundles have same usedBy elements.
>>> sameUE: it menas the number of two bundles have same usedExternal
>>> external package rely elements.
>>> 
>>> boolean condition1 = 4 * sameUB >= one.usedByList.size() +
>>> two.usedByList.size();
>>> boolean condition2 = 2 * sameUE >= one.usedExternalList.size() +
>>> two.usedExternalList.size();
>>> 
>>> if condition1 or condition2 is true, we should merge these two usedBy
>>> only bundle.
>>> 
>>> Merge the other bundles:
>>> 5.The core problem is how to decide two bundles(bundle one and bundle
>>> two) can be merged or not.
>>> 
>>> Define 5 variables:
>>> uiNumber: the sum of bundle one's usesInternal elements in bundle two
>>> number and bundle two's usesInternal elements in bundle one number;
>>> 
>>> ubNumber:the sum of bundle one's usedBy elements in bundle two number
>>> and bundle two's usedBy elements in bundle one number;
>>> 
>>> sameUI: the same usesInternal number bundle one and two have
>>> 
>>> sameUB: Be similar with used by only bundle's this variable
>>> 
>>> sameUE: Be similar with used by only bundle's this variable
>>> 
>>> Define these conditions:
>>> boolean condition1 = 2 * uiNumber >= one.usesInternalList.size() +
>>> two.usesInternalList.size();
>>> boolean condition2 = 2 * ubNumber >= one.usedByList.size() +
>>> two.usedByList.size() ;
>>> boolean condition3 = 3.5 * sameUI >= one.usesInternalList.size() +
>>> two.usesInternalList.size();
>>> boolean condition4 = 4 * sameUB >= one.usedByList.size() +
>>> two.usedByList.size();
>>> boolean condition5 = 3 * sameUE >= one.usedExternalList.size() +
>>> two.usedExternalList.size();
>>> 
>>> If any above condition is true, these two bundles can be merged. But
>>> these are another problem, if bundle A can be merged with B, but it
>>> also can be merged with C, now, we should decide merge A with B or A
>>> with C.
>>> 
>>> Define the follow attribute:
>>> int mergeFactor= 2 * (uiNumber + ubNumber) + 0.4 * (sameUI + sameUB) +
>>> 0.2 * sameUE - number;
>>> 
>>> If A and B's mergeFactor is 20 and A and C's mergeFactor is 30, we
>>> should merge A and C.
>>> 
>>> This is current merge algorithm in OSGiMaker, i will keep on improving
>>> it, use class relationship factors or etc. You can find the source
>>> code detail of this algorithm in class AnalyseJob of our project.
>>> 
>>> The attach file is the analyse result document OSGiMaker analyse
>>> Tomcat's source code and split it into bundles, you can have a review.
>>> 
>>> 
>>> In fact, i did not want to bother you too much, but it seems that you
>>> have enough time to help me to improve it, this is a good thing. If
>>> you have any advises, please let me know, let's improving it together
>>> :-)
>>> 
>>> Thank you
>>> 
>>> [1] http://code.google.com/p/osgimaker/wiki/Implement_of_project_analyse_algorithm
>>> 
>>> 2011/7/26 Peter Kriens <peter.kriens@aqute.biz>:
>>>> Well, I do not think I am eager but I have a hard time getting a feeling
where you are. You do not have to send reports, I want to see intermediate results and discuss
issues if there are any. As I said earlier, it is not clear to me yet what the best algorithm
is so that need to be worked out before we do the gui stuff.
>>>> 
>>>> Kind regards,
>>>> 
>>>>        Peter Kriens
>>>> 
>>>> 
>>>> 
>>>> On 26 jul 2011, at 11:24, Tiger Gui wrote:
>>>> 
>>>>> In my schedule, i will report current status to you tomorrow as i
>>>>> think i can get a usable version today,  the whole analyse and split
>>>>> algorithm is complex i have to organize a document to describe it
>>>>> clearly. As you are really eager to see its progress, it is OK, i will
>>>>> start the report now
>>>>> 
>>>>> 2011/7/26 Peter Kriens <peter.kriens@aqute.biz>:
>>>>>> If that is the case you have to provide more on going feedback. What
happened to the analysis by hand?
>>>>>> 
>>>>>> Kind regards,
>>>>>> 
>>>>>>        Peter Kriens
>>>>>> 
>>>>>> 
>>>>>> On 26 jul 2011, at 11:02, Tiger Gui wrote:
>>>>>> 
>>>>>>> No, Peter, i am really working hard for this project, you can
check
>>>>>>> the progress here[1]. Now, this tool can analyse a whole project
and
>>>>>>> export the its analyse result to bundles( I test it in Spring
and
>>>>>>> Tomcat project), if possible, you can install it in your Eclipse
and
>>>>>>> have a trial of it. But, i am still improving the split algorithm
as
>>>>>>> the current algorithm is not working perfect.
>>>>>>> 
>>>>>>> I will supply a document about the current status of this project
and
>>>>>>> a simple guide for you to have a trial of it. I am really working
very
>>>>>>> hard for it these days :-(
>>>>>>> 
>>>>>>> [1] http://code.google.com/p/osgimaker/updates/list
>>>>>>> 
>>>>>>> 2011/7/26 Peter Kriens <peter.kriens@aqute.biz>:
>>>>>>>> I am getting the feeling that you're not working very hard
on this project and only does something just for the evaluations ...
>>>>>>>> 
>>>>>>>>        Peter Kriens
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Best Regards
>>>>>>> ----------------------------------------------------
>>>>>>> Tiger Gui [tigergui1990@gmail.com]
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Best Regards
>>>>> ----------------------------------------------------
>>>>> Tiger Gui [tigergui1990@gmail.com]
>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Best Regards
>>> ----------------------------------------------------
>>> Tiger Gui [tigergui1990@gmail.com]
>>> <TomcatJava.analyse>
>> 
>> 
> 
> 
> 
> -- 
> Best Regards
> ----------------------------------------------------
> Tiger Gui [tigergui1990@gmail.com]
> <bundles_relation.txt><SpringSplitTest.analyse>


Mime
View raw message