airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amila Jayasekara <thejaka.am...@gmail.com>
Subject Re: Improvements to Experiment input data model in order to support Gaussian application
Date Wed, 10 Dec 2014 18:30:21 GMT
Also, regarding the handler that Shameera is working on ...
I guess that handler is going to change mainly "qsub" parameters or "aprun"
parameters (Correct me if i am wrong). I think it would be more useful to
write a handler which changes any parameter in "qsub", "aprun" or
"mpiexec".

In implementation wise I would imagine there is an abstract handler with
concrete implementation for each job scheduling command.

Thanks
-Amila

On Wed, Dec 10, 2014 at 9:17 AM, Marlon Pierce <marpierc@iu.edu> wrote:

> +1 for more generalization.
>
> We are collecting more raw material for chemistry application use cases at
> https://cwiki.apache.org/confluence/display/AIRAVATA/Use+Cases. We'll
> review them (and bio apps that we also collected previously) in a wiki
> document to see if our API mappings are correct.
>
> Preliminarily, we see the command line arguments don't contain the full
> list of input and output files.  Additional required inputs may be passed
> via control files, environment variables, etc.  Examples include data
> libraries for basis functions, names of checkpoint files, names of output
> files, and so forth.  So we need a way to say the application may take 4
> inputs, but only 1 is needed to construct a valid command line, for example.
>
> On the other hand, I don't think we need the InputMetadataType that
> Chathuri introduces below. This overlaps with what is already in the
> compute resource description fields.
>
>
> Marlon
>
>
> On 12/8/14, 10:17 PM, Amila Jayasekara wrote:
>
>> Hi Chathuri,
>>
>> I do not know anything about Gaussian. So its kind of hard for me to
>> understand what exactly is the meaning of the structures you introduced
>> and
>> why you exactly need those structures.
>>
>> A more important question is how to come up with a more abstract and
>> generic thrift IDLS so that you dont need to change it every time we add a
>> new application. Going through many example applications is certainly a
>> good way to understand broad requirements and helps to abstract out many
>> features.
>>
>> Thanks
>> -Thejaka
>>
>> On Mon, Dec 8, 2014 at 10:22 AM, Chathuri Wimalasena <
>> kamalasini@gmail.com>
>> wrote:
>>
>>  Hi Devs,
>>>
>>> We are trying to add Gaussian application using airavata-appcatalog.
>>> While
>>> doing that, we face some limitations of the current design.
>>>
>>> In Gaussian there are several input files, some input files should used
>>> when the job run command is generated, but some does not.  Those which
>>> are
>>> not involved with job run command also need to be staged to working
>>> directory. Such flags are not supported in current design.
>>>
>>> Another interesting feature that in Gaussian is, in input file, we can
>>> specify the values for memory, cpu like options. If input file includes
>>> those parameters, we need to give priority to those values instead of the
>>> values specified in the request.
>>>
>>> To support these features, we need to slightly modify our thrift IDLS,
>>> specially to InputDataObjectType struct.
>>>
>>> Current struct is below.
>>>
>>> struct InputDataObjectType {
>>>      1: required string name,
>>>      2: optional string value,
>>>      3: optional DataType type,
>>>      4: optional string applicationArgument,
>>>      5: optional bool standardInput = 0,
>>>      6: optional string userFriendlyDescription,
>>>      7: optional string metaData
>>> }
>>>
>>> In order to support 1st requirement, we introduce 2 enums.
>>>
>>> enum InputValidityType{
>>> REQUIRED,
>>> OPTIONAL
>>> }
>>>
>>> enum CommandLineType{
>>> INCLUSIVE,
>>> EXCLUSIVE
>>> }
>>>
>>> Please excuse me for names. You are welcome to suggest better names.
>>>
>>> To support 2nd requirement, we change metaData field to a map with
>>> another
>>> enum where we define all the metadata types that can have.
>>>
>>> enum InputMetadataType {
>>>      MEMORY,
>>>      CPU
>>> }
>>>
>>> So the new InputDataObjectType would be as below.
>>>
>>> struct InputDataObjectType {
>>>      1: required string name,
>>>      2: optional string value,
>>>      3: optional DataType type,
>>>      4: optional string applicationArgument,
>>>      5: optional bool standardInput = 0,
>>>      6: optional string userFriendlyDescription,
>>>    *  7: optional map<InputMetadataType, string> metaData,*
>>> *    8: optional InputValidityType inputValid;*
>>> *    9: optional CommandLineType addedToCommandLine;*
>>> *    10: optional bool dataStaged = 0;*
>>> }
>>>
>>> Suggestions are welcome.
>>>
>>> Thanks,
>>> Chathuri
>>>
>>>
>>>
>

Mime
View raw message