airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Gunathilake <glah...@gmail.com>
Subject Re: Suggestions on maintaining the server/client configuration files?
Date Mon, 31 Mar 2014 13:40:14 GMT
I think we already have it in gfac-schemas and using tool like this[1], we
can quickly generate forms without worrying about changing them when the
schema changes.

But I think we are going to do a new Application Catalog and once its done
we can use some tool to do the same thing.

[1]https://code.google.com/p/xsd-web-forms/
Regards
Lahiru


On Mon, Mar 31, 2014 at 9:31 AM, Sachith Withana <swsachith@gmail.com>wrote:

> Why don't we have some kind of Util module to store all these config files
> and have a rest-kind of class which acts as the middle layer between
> airavata modules and config files?
> All the config files are in one place, have a programmatic method of
> accessing them.
>
> IMO we should have a different set of config files for the Integration
> tests,
> 1. It would test the system using the thrift 'client' using external
> configurations files
> 2. since it is a main entry point for a gateway/airavata dev, it will show
> a clearer image on  which configurations we should use when we are using
> the client (it would only be the URLs and the ports for now).
>  3. It would eliminate any airavata-internal dependencies for the
> integration tests.
>
>
> On Mon, Mar 31, 2014 at 1:16 AM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>
>> It depends upon the test. If someone adds or updates configuration data
>> related to the correct functioning of the system or a component it is very
>> likely that those configuration needs to be present for unit tests as well
>> in case the unit tests depends upon those other components. Thus it becomes
>> necessary to update all the test configuration files (to be on the safe
>> side).
>>  On Mar 30, 2014 8:29 PM, "Lahiru Gunathilake" <glahiru@gmail.com> wrote:
>>
>>> Hi Saminda,
>>>
>>> Ideally even at this point we should be able to remove all the
>>> configuration files from src/main/resources but for tests we might need the
>>> files in src/test/resources.
>>>
>>> good thing about this is we have all the configuration in one place
>>> (airavata-server.properties) but bad thing is we have to keep it in each
>>> module's test configuration.
>>>
>>> Is this really a problem ?
>>>
>>> Regards
>>> Lahiru
>>>
>>>
>>> On Sun, Mar 30, 2014 at 9:59 PM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>>>
>>>> In another related note I found few places which has additional
>>>> property file types present in our trunk, some examples are,
>>>>
>>>> tools/job-monitor/src/main/resources/gsissh.properties
>>>> tools/job-monitor/src/test/resources/monitor.properties
>>>> tools/phoebus-integration/src/main/resources/service.properties
>>>> modules/commons/gfac-schema/src/main/resources/datatype.properties
>>>> modules/commons/workflow-tracking/src/main/resources/log4j.properties
>>>>
>>>> modules/orchestrator/airavata-orchestrator-service/src/test/resources/orchestrator.properties
>>>>
>>>> modules/ws-messenger/messagebroker/src/test/resources/unit_tests.properties
>>>> modules/test-suite/src/test/resources/unit_test.properties
>>>> modules/xbaya-gui/src/main/resources/airavata-client.properties
>>>> modules/xbaya-gui/src/test/resources/airavata-server.properties
>>>> modules/gfac/gfac-core/src/main/resources/errors.properties
>>>> modules/gfac/gfac-core/src/main/resources/service.properties
>>>> modules/gfac/gfac-core/src/test/resources/logging.properties
>>>> modules/gfac/gfac-core/src/test/resources/airavata-client.properties
>>>>
>>>> I remember we raised an issue relating to having component property
>>>> files but not coming to a conclusion regarding it. Any thoughts about it?
>>>>
>>>>
>>>> On Thu, Mar 27, 2014 at 9:23 PM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>>>>
>>>>> One way which so far seems to work is to define 2 maven modules
>>>>> airavata-server-configuration and airavata-client-configurations as jar
>>>>> artifacts where each will have the respective configuration files. Then
use
>>>>> them as dependencies for the modules which require the relevant
>>>>> configurations. For the binary distributions we will have to use the
>>>>> assembly-plugin and/or the dependency-plugin to copy the configuration
>>>>> files in to the correct location in the distribution directory.
>>>>>
>>>>> I'm still working on how this solution can be adhered when running
>>>>> unit tests. At the moment the the duplicated configuration files in
>>>>> src/test/resources only has a few changes compared to the distributed
>>>>> configurations. Any thoughts?
>>>>>
>>>>>
>>>>> On Wed, Mar 26, 2014 at 10:22 PM, Saminda Wijeratne <
>>>>> samindaw@gmail.com> wrote:
>>>>>
>>>>>> A little update on the efforts on this...
>>>>>>
>>>>>>    - Like Amila suggested, ServerSettings now by default supports
>>>>>>    parameterization
>>>>>>    - settings can be added/updated by passing the new settings via
>>>>>>    command line (and incidentally passing them to the static func.
>>>>>>    ServerMain.main(...)).
>>>>>>       - $ ./airavata-server.sh --myproxy.user=saminda
>>>>>>       --myproxy.pass=pass123
>>>>>>       - Introduced ServerSettings.mergeSettings...(...) functions
to
>>>>>>    help add/update settings through Maps, command line args, additional
prop
>>>>>>    files.
>>>>>>
>>>>>> The issue that I'm still facing is the duplication of the
>>>>>> airavata-server.properties configuration file in the code base. Last
time I
>>>>>> checked copies of it is spread out at 14 locations in the trunk (for
>>>>>> running junit tests, in standalone server dist, in integration tests,
in
>>>>>> sample gateway).
>>>>>>
>>>>>>    - config files in a single location in the trunk and refer to
>>>>>>    them using relative paths:
>>>>>>       - done using the maven-builder-helper plugin.
>>>>>>       - But it seems the IDEs doesn't recognize resources outside
>>>>>>       the project folder.
>>>>>>    - "git subtrees" (which allows two way code sharing) similar to
>>>>>>    "svn externals":
>>>>>>    - the limitations in it (configs needs to be in a different repo,
>>>>>>       subtree will be merged on the root, cannot specifically select
which code
>>>>>>       to share?), it seems its not a viable option.
>>>>>>
>>>>>> any ideas?
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 6, 2014 at 1:55 PM, Amila Jayasekara <
>>>>>> thejaka.amila@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Saminda,
>>>>>>>
>>>>>>> I guess the best thing is to create template configuration files
in
>>>>>>> a central location and replace needed variables within the test
>>>>>>> initialization and place generated config files within the target
directory
>>>>>>> of the test case.
>>>>>>>
>>>>>>> E.g :- registry.url = ${registry.url}
>>>>>>>
>>>>>>> ${registry.url} is replaced within the test case with appropriate
>>>>>>> value. Then we dont need to duplicate configuration files in
everywhere.
>>>>>>> Also when a configuration is updated we only need to change in
a single
>>>>>>> place.
>>>>>>> You may encapsulate template parameter replacing logic into a
common
>>>>>>> util class so that each test case can just invoke the logic.
>>>>>>>
>>>>>>> Further we should have a single place to read all configurations
and
>>>>>>> all subcomponents must go through this configuration component
to get
>>>>>>> config values. Otherwise it will be hard to maintain the configuration
>>>>>>> reading code.
>>>>>>>
>>>>>>> Thanks
>>>>>>> Amila
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 6, 2014 at 1:38 PM, Saminda Wijeratne <
>>>>>>> samindaw@gmail.com> wrote:
>>>>>>>
>>>>>>>> There are several places which we need the
>>>>>>>> airavata-server.properties and airavata-client.properties
files to be
>>>>>>>> present in order to run tests and standalone servers resulting
in
>>>>>>>> replication. Any idea how we should handle this?
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> System Analyst Programmer
>>> PTI Lab
>>> Indiana University
>>>
>>
>
>
> --
> Thanks,
> Sachith Withana
>
>


-- 
System Analyst Programmer
PTI Lab
Indiana University

Mime
View raw message