airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sachith Withana <swsach...@gmail.com>
Subject Re: Suggestions on maintaining the server/client configuration files?
Date Mon, 31 Mar 2014 13:31:05 GMT
Why don't we have some kind of Util module to store all these config files
and have a rest-kind of class which acts as the middle layer between
airavata modules and config files?
All the config files are in one place, have a programmatic method of
accessing them.

IMO we should have a different set of config files for the Integration
tests,
1. It would test the system using the thrift 'client' using external
configurations files
2. since it is a main entry point for a gateway/airavata dev, it will show
a clearer image on  which configurations we should use when we are using
the client (it would only be the URLs and the ports for now).
 3. It would eliminate any airavata-internal dependencies for the
integration tests.


On Mon, Mar 31, 2014 at 1:16 AM, Saminda Wijeratne <samindaw@gmail.com>wrote:

> It depends upon the test. If someone adds or updates configuration data
> related to the correct functioning of the system or a component it is very
> likely that those configuration needs to be present for unit tests as well
> in case the unit tests depends upon those other components. Thus it becomes
> necessary to update all the test configuration files (to be on the safe
> side).
> On Mar 30, 2014 8:29 PM, "Lahiru Gunathilake" <glahiru@gmail.com> wrote:
>
>> Hi Saminda,
>>
>> Ideally even at this point we should be able to remove all the
>> configuration files from src/main/resources but for tests we might need the
>> files in src/test/resources.
>>
>> good thing about this is we have all the configuration in one place
>> (airavata-server.properties) but bad thing is we have to keep it in each
>> module's test configuration.
>>
>> Is this really a problem ?
>>
>> Regards
>> Lahiru
>>
>>
>> On Sun, Mar 30, 2014 at 9:59 PM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>>
>>> In another related note I found few places which has additional property
>>> file types present in our trunk, some examples are,
>>>
>>> tools/job-monitor/src/main/resources/gsissh.properties
>>> tools/job-monitor/src/test/resources/monitor.properties
>>> tools/phoebus-integration/src/main/resources/service.properties
>>> modules/commons/gfac-schema/src/main/resources/datatype.properties
>>> modules/commons/workflow-tracking/src/main/resources/log4j.properties
>>>
>>> modules/orchestrator/airavata-orchestrator-service/src/test/resources/orchestrator.properties
>>>
>>> modules/ws-messenger/messagebroker/src/test/resources/unit_tests.properties
>>> modules/test-suite/src/test/resources/unit_test.properties
>>> modules/xbaya-gui/src/main/resources/airavata-client.properties
>>> modules/xbaya-gui/src/test/resources/airavata-server.properties
>>> modules/gfac/gfac-core/src/main/resources/errors.properties
>>> modules/gfac/gfac-core/src/main/resources/service.properties
>>> modules/gfac/gfac-core/src/test/resources/logging.properties
>>> modules/gfac/gfac-core/src/test/resources/airavata-client.properties
>>>
>>> I remember we raised an issue relating to having component property
>>> files but not coming to a conclusion regarding it. Any thoughts about it?
>>>
>>>
>>> On Thu, Mar 27, 2014 at 9:23 PM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>>>
>>>> One way which so far seems to work is to define 2 maven modules
>>>> airavata-server-configuration and airavata-client-configurations as jar
>>>> artifacts where each will have the respective configuration files. Then use
>>>> them as dependencies for the modules which require the relevant
>>>> configurations. For the binary distributions we will have to use the
>>>> assembly-plugin and/or the dependency-plugin to copy the configuration
>>>> files in to the correct location in the distribution directory.
>>>>
>>>> I'm still working on how this solution can be adhered when running unit
>>>> tests. At the moment the the duplicated configuration files in
>>>> src/test/resources only has a few changes compared to the distributed
>>>> configurations. Any thoughts?
>>>>
>>>>
>>>> On Wed, Mar 26, 2014 at 10:22 PM, Saminda Wijeratne <samindaw@gmail.com
>>>> > wrote:
>>>>
>>>>> A little update on the efforts on this...
>>>>>
>>>>>    - Like Amila suggested, ServerSettings now by default supports
>>>>>    parameterization
>>>>>    - settings can be added/updated by passing the new settings via
>>>>>    command line (and incidentally passing them to the static func.
>>>>>    ServerMain.main(...)).
>>>>>       - $ ./airavata-server.sh --myproxy.user=saminda
>>>>>       --myproxy.pass=pass123
>>>>>       - Introduced ServerSettings.mergeSettings...(...) functions to
>>>>>    help add/update settings through Maps, command line args, additional
prop
>>>>>    files.
>>>>>
>>>>> The issue that I'm still facing is the duplication of the
>>>>> airavata-server.properties configuration file in the code base. Last
time I
>>>>> checked copies of it is spread out at 14 locations in the trunk (for
>>>>> running junit tests, in standalone server dist, in integration tests,
in
>>>>> sample gateway).
>>>>>
>>>>>    - config files in a single location in the trunk and refer to them
>>>>>    using relative paths:
>>>>>       - done using the maven-builder-helper plugin.
>>>>>       - But it seems the IDEs doesn't recognize resources outside the
>>>>>       project folder.
>>>>>    - "git subtrees" (which allows two way code sharing) similar to
>>>>>    "svn externals":
>>>>>    - the limitations in it (configs needs to be in a different repo,
>>>>>       subtree will be merged on the root, cannot specifically select
which code
>>>>>       to share?), it seems its not a viable option.
>>>>>
>>>>> any ideas?
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Mar 6, 2014 at 1:55 PM, Amila Jayasekara <
>>>>> thejaka.amila@gmail.com> wrote:
>>>>>
>>>>>> Hi Saminda,
>>>>>>
>>>>>> I guess the best thing is to create template configuration files
in a
>>>>>> central location and replace needed variables within the test
>>>>>> initialization and place generated config files within the target
directory
>>>>>> of the test case.
>>>>>>
>>>>>> E.g :- registry.url = ${registry.url}
>>>>>>
>>>>>> ${registry.url} is replaced within the test case with appropriate
>>>>>> value. Then we dont need to duplicate configuration files in everywhere.
>>>>>> Also when a configuration is updated we only need to change in a
single
>>>>>> place.
>>>>>> You may encapsulate template parameter replacing logic into a common
>>>>>> util class so that each test case can just invoke the logic.
>>>>>>
>>>>>> Further we should have a single place to read all configurations
and
>>>>>> all subcomponents must go through this configuration component to
get
>>>>>> config values. Otherwise it will be hard to maintain the configuration
>>>>>> reading code.
>>>>>>
>>>>>> Thanks
>>>>>> Amila
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 6, 2014 at 1:38 PM, Saminda Wijeratne <samindaw@gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> There are several places which we need the
>>>>>>> airavata-server.properties and airavata-client.properties files
to be
>>>>>>> present in order to run tests and standalone servers resulting
in
>>>>>>> replication. Any idea how we should handle this?
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> System Analyst Programmer
>> PTI Lab
>> Indiana University
>>
>


-- 
Thanks,
Sachith Withana

Mime
View raw message