airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saminda Wijeratne <samin...@gmail.com>
Subject Re: Suggestions on maintaining the server/client configuration files?
Date Fri, 04 Apr 2014 09:18:38 GMT
hi devs,

I refactored the trunk code to link airavata-server.properties and
airavata-client.properties from 2 maven modules now included in the trunk.
modules/configurations/server
modules/configurations/client

airavata-server.properties is used by the "airavata-standalone-server"
module (@ modules/server) and in the server distribution.
airavata-client.properties was used in,
modules/orchestrator/orchestrator-client-sdks
modules/xbaya-gui
modules/distribution/xbaya-gui
modules/distribution/airavata-client
modules/airavata-client
airavata-api/airavata-client-sdks/java-client-samples

However I was not able to carry out similar merge for the properties files
used in the tests since I kept hitting on broken unit tests. I will try to
fix them first along with the issue reported by lahiru [2].

Saminda

1. https://issues.apache.org/jira/browse/AIRAVATA-1121
2. https://issues.apache.org/jira/browse/AIRAVATA-1119



On Mon, Mar 31, 2014 at 6:40 AM, Lahiru Gunathilake <glahiru@gmail.com>wrote:

> I think we already have it in gfac-schemas and using tool like this[1], we
> can quickly generate forms without worrying about changing them when the
> schema changes.
>
> But I think we are going to do a new Application Catalog and once its done
> we can use some tool to do the same thing.
>
> [1]https://code.google.com/p/xsd-web-forms/
> Regards
> Lahiru
>
>
> On Mon, Mar 31, 2014 at 9:31 AM, Sachith Withana <swsachith@gmail.com>wrote:
>
>> Why don't we have some kind of Util module to store all these config
>> files and have a rest-kind of class which acts as the middle layer between
>> airavata modules and config files?
>> All the config files are in one place, have a programmatic method of
>> accessing them.
>>
>> IMO we should have a different set of config files for the Integration
>> tests,
>> 1. It would test the system using the thrift 'client' using external
>> configurations files
>> 2. since it is a main entry point for a gateway/airavata dev, it will
>> show a clearer image on  which configurations we should use when we are
>> using the client (it would only be the URLs and the ports for now).
>>  3. It would eliminate any airavata-internal dependencies for the
>> integration tests.
>>
>>
>> On Mon, Mar 31, 2014 at 1:16 AM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>>
>>> It depends upon the test. If someone adds or updates configuration data
>>> related to the correct functioning of the system or a component it is very
>>> likely that those configuration needs to be present for unit tests as well
>>> in case the unit tests depends upon those other components. Thus it becomes
>>> necessary to update all the test configuration files (to be on the safe
>>> side).
>>>  On Mar 30, 2014 8:29 PM, "Lahiru Gunathilake" <glahiru@gmail.com>
>>> wrote:
>>>
>>>> Hi Saminda,
>>>>
>>>> Ideally even at this point we should be able to remove all the
>>>> configuration files from src/main/resources but for tests we might need the
>>>> files in src/test/resources.
>>>>
>>>> good thing about this is we have all the configuration in one place
>>>> (airavata-server.properties) but bad thing is we have to keep it in each
>>>> module's test configuration.
>>>>
>>>> Is this really a problem ?
>>>>
>>>> Regards
>>>> Lahiru
>>>>
>>>>
>>>> On Sun, Mar 30, 2014 at 9:59 PM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>>>>
>>>>> In another related note I found few places which has additional
>>>>> property file types present in our trunk, some examples are,
>>>>>
>>>>> tools/job-monitor/src/main/resources/gsissh.properties
>>>>> tools/job-monitor/src/test/resources/monitor.properties
>>>>> tools/phoebus-integration/src/main/resources/service.properties
>>>>> modules/commons/gfac-schema/src/main/resources/datatype.properties
>>>>> modules/commons/workflow-tracking/src/main/resources/log4j.properties
>>>>>
>>>>> modules/orchestrator/airavata-orchestrator-service/src/test/resources/orchestrator.properties
>>>>>
>>>>> modules/ws-messenger/messagebroker/src/test/resources/unit_tests.properties
>>>>> modules/test-suite/src/test/resources/unit_test.properties
>>>>> modules/xbaya-gui/src/main/resources/airavata-client.properties
>>>>> modules/xbaya-gui/src/test/resources/airavata-server.properties
>>>>> modules/gfac/gfac-core/src/main/resources/errors.properties
>>>>> modules/gfac/gfac-core/src/main/resources/service.properties
>>>>> modules/gfac/gfac-core/src/test/resources/logging.properties
>>>>> modules/gfac/gfac-core/src/test/resources/airavata-client.properties
>>>>>
>>>>> I remember we raised an issue relating to having component property
>>>>> files but not coming to a conclusion regarding it. Any thoughts about
it?
>>>>>
>>>>>
>>>>> On Thu, Mar 27, 2014 at 9:23 PM, Saminda Wijeratne <samindaw@gmail.com
>>>>> > wrote:
>>>>>
>>>>>> One way which so far seems to work is to define 2 maven modules
>>>>>> airavata-server-configuration and airavata-client-configurations
as jar
>>>>>> artifacts where each will have the respective configuration files.
Then use
>>>>>> them as dependencies for the modules which require the relevant
>>>>>> configurations. For the binary distributions we will have to use
the
>>>>>> assembly-plugin and/or the dependency-plugin to copy the configuration
>>>>>> files in to the correct location in the distribution directory.
>>>>>>
>>>>>> I'm still working on how this solution can be adhered when running
>>>>>> unit tests. At the moment the the duplicated configuration files
in
>>>>>> src/test/resources only has a few changes compared to the distributed
>>>>>> configurations. Any thoughts?
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 26, 2014 at 10:22 PM, Saminda Wijeratne <
>>>>>> samindaw@gmail.com> wrote:
>>>>>>
>>>>>>> A little update on the efforts on this...
>>>>>>>
>>>>>>>    - Like Amila suggested, ServerSettings now by default supports
>>>>>>>    parameterization
>>>>>>>    - settings can be added/updated by passing the new settings
via
>>>>>>>    command line (and incidentally passing them to the static
func.
>>>>>>>    ServerMain.main(...)).
>>>>>>>       - $ ./airavata-server.sh --myproxy.user=saminda
>>>>>>>       --myproxy.pass=pass123
>>>>>>>       - Introduced ServerSettings.mergeSettings...(...) functions
>>>>>>>    to help add/update settings through Maps, command line args,
additional
>>>>>>>    prop files.
>>>>>>>
>>>>>>> The issue that I'm still facing is the duplication of the
>>>>>>> airavata-server.properties configuration file in the code base.
Last time I
>>>>>>> checked copies of it is spread out at 14 locations in the trunk
(for
>>>>>>> running junit tests, in standalone server dist, in integration
tests, in
>>>>>>> sample gateway).
>>>>>>>
>>>>>>>    - config files in a single location in the trunk and refer
to
>>>>>>>    them using relative paths:
>>>>>>>       - done using the maven-builder-helper plugin.
>>>>>>>       - But it seems the IDEs doesn't recognize resources outside
>>>>>>>       the project folder.
>>>>>>>    - "git subtrees" (which allows two way code sharing) similar
to
>>>>>>>    "svn externals":
>>>>>>>    - the limitations in it (configs needs to be in a different
>>>>>>>       repo, subtree will be merged on the root, cannot specifically
select which
>>>>>>>       code to share?), it seems its not a viable option.
>>>>>>>
>>>>>>> any ideas?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 6, 2014 at 1:55 PM, Amila Jayasekara <
>>>>>>> thejaka.amila@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Saminda,
>>>>>>>>
>>>>>>>> I guess the best thing is to create template configuration
files in
>>>>>>>> a central location and replace needed variables within the
test
>>>>>>>> initialization and place generated config files within the
target directory
>>>>>>>> of the test case.
>>>>>>>>
>>>>>>>> E.g :- registry.url = ${registry.url}
>>>>>>>>
>>>>>>>> ${registry.url} is replaced within the test case with appropriate
>>>>>>>> value. Then we dont need to duplicate configuration files
in everywhere.
>>>>>>>> Also when a configuration is updated we only need to change
in a single
>>>>>>>> place.
>>>>>>>> You may encapsulate template parameter replacing logic into
a
>>>>>>>> common util class so that each test case can just invoke
the logic.
>>>>>>>>
>>>>>>>> Further we should have a single place to read all configurations
>>>>>>>> and all subcomponents must go through this configuration
component to get
>>>>>>>> config values. Otherwise it will be hard to maintain the
configuration
>>>>>>>> reading code.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Amila
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 6, 2014 at 1:38 PM, Saminda Wijeratne <
>>>>>>>> samindaw@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> There are several places which we need the
>>>>>>>>> airavata-server.properties and airavata-client.properties
files to be
>>>>>>>>> present in order to run tests and standalone servers
resulting in
>>>>>>>>> replication. Any idea how we should handle this?
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> System Analyst Programmer
>>>> PTI Lab
>>>> Indiana University
>>>>
>>>
>>
>>
>> --
>> Thanks,
>> Sachith Withana
>>
>>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
>

Mime
View raw message