airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Marru <sma...@apache.org>
Subject Re: Too Many Leaf Modules.
Date Sat, 30 May 2015 03:56:13 GMT
Hi Shameera,

I am reading your concerns properly now. I agree on re-organizing the modules to eliminate
some of the unnecessary organization. I also agree we need to right away address the issues
of decoupling interfaces and implementations. 

I do not think bundling all core into one module is a good idea. When there is distinct enough
functionality why combine? There are advantages with micro-service architecture both in terms
of rapid evolution as well as ease of light weight deployments. For instance lets take API
server and Orchestrator example. API server gets hit for all the calls, but only a fraction
of them are intended for Orchestrator. Even in your approach of starting different services
from one module, the dependencies are all class loaded (unless we put sophistication in startup
scripts). 

If two modules have significant overlapping functionality then + 1 to discuss those as case
by case and combine them as necessary. But can you explain a little more on why you want to
combine distinct functional modules? 

Suresh

> On May 29, 2015, at 11:29 AM, Suresh Marru <smarru@apache.org> wrote:
> 
> + 1. 
> 
> I was planning to bring up this issue also. Probably it will not address what you are
raising, but here is a tree output from airavata labs code I was toying with locally. I did
not yet compare it with what you proposed, I will do so later today.
> 
> ├── airavata-api
> │   ├── airavata-api-interface-descriptions
> │   ├── airavata-api-java-stubs
> │   ├── airavata-api-server
> │   ├── airavata-data-models
> │   ├── api-security-manager
> ├── clients
> │   ├── airavata-client-cpp-sdk
> │   ├── airavata-client-java-sdk
> │   ├── airavata-client-php-sdk
> │   ├── airavata-client-python-sdk
> │   ├── airavata-sample-examples
> │   └── airavata-xbaya-gui
> ├── components
> │   ├── commons
> │   ├── component-interface-descriptions
> │   ├── component-services
> │   │   ├── credential-store-service
> │   │   ├── orchestrator-service
> │   │   ├── task-executor-service
> │   │   └── workflow-interpreter-service
> │   ├── component-clients
> │   │   ├── credential-store-client
> │   │   ├── orchestrator-client
> │   │   ├── task-executor-client
> │   │   ├── workflow-interpreter-client
> │   │   └── messaging
> │   ├── task-adaptors
> │   │   ├── compute
> │   │   └── data-movement
> │   ├── registry
> │   │   ├── app-catalog
> │   │   ├── experiment-catalog
> │   │   └── resource-catalog
> │   └── workflow-interpreter
> ├── distribution
> ├── integration-tests
> 
> 
> 
>> On May 29, 2015, at 10:15 AM, Shameera Rathnayaka <shameera@apache.org <mailto:shameera@apache.org>>
wrote:
>> 
>> Hi Devs, 
>> 
>> As we are using different modules to package different type of functionalities, which
will help us to maintain loosely couple codes. Now the project has 49 leaf module ( one to
hit half century :) ). If we allow project to grow this way, having too fine grain modules
will be huge headache in future. IMO we should clean this ASAP before it become really mess.
Actually we half way there, I experienced cyclic dependency issues when I was writing workflow
implementation and email monitoring. Please see the modules in current repo below. 
>> 
>> <module-name> ( <num of child modules> )
>> 
>> modules  ( 43 )
>>      app-catalog ( 2 )
>>      commons ( 1 )
>>      configurations ( 2 )
>>      credential-store ( 3 )
>>      distribution ( 8 )
>>      gfac ( 10 )
>>      integration test ( 1 )
>>      messaging ( 2 )
>>      orchestrator ( 3 )
>>      registry ( 3 )
>>      security ( 1 )
>>      server ( 1 )
>>      test-suit ( 1 )
>>      workflow ( 1 )
>>      workflow-modal ( 3 )
>>      xbaya ( 1 ) 
>> airavata-api ( 5 )
>> tools ( 1 ) 
>> 
>> Most of the current modules have interfaces and implementations together, but this
violate our main goal which reduce inter module dependencies. Following is what I am suggesting,
WDYS?
>> 
>> core { has all core interfaces and basic classes of gfac-core , orchestrator-core
, message-core , monitor core, registry core, workflow-core}
>> service - all thrift services and service handlers 
>> orchestrator - orchestrator specific classes
>> gfac 
>>      SSH  
>>      BES
>>      Local
>> message - amqp implemention 
>> distribution 
>>      XBaya
>>      server - { use different mode input to start server as orchestrator , Gfac or/and
api-server }
>> commons
>> registry
>> app-catalog
>> security
>> Workflow
>> XBaya-gui
>> Integration-test 
>> 
>> Thanks, 
>> Shameera.
> 


Mime
View raw message