airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Gunathilake <glah...@gmail.com>
Subject Re: Application Catalog Design Phase-I
Date Wed, 07 May 2014 14:22:16 GMT
Here are my thoughts,

Job properties are configured as key value pairs, if we do that gateway
developer has to manually look in to the keys what we internally used to
store certain data and use them because during the validation steps and
provider level steps we will be using defined set of job properties. I
think it will be a good idea to specify heavily used properties as members
of a JobProperties class and always support key/value pairs at the end or
if we are moving towards key/value pair make sure you define Enums for each
type of job properties you we supports so that gateway developer can use
them.

I think the hierarchy can looks like below.

host - simply the address (trestles.scsc.edu, id)
|
|--* Job Manager (contains job manager properties )
     (torque - path - /bin/qsub some other properties and always support
key/value pairs)
|
|-  *Monitor Manager
     (qstat - path = /bin/qstat, type = pull some other properties)

| - key value pairs for the host (can be used if someone wants to put their
own property)


Deployment
|
|- hostID (Just the ID)

|- Other deployement specific properties


During the scheduling process we can pick a single deployment then get the
hostid and select a  job manager and monitor manager based on some logic.
Each host will have default values to be picked if scheduling is disabled.


Thanks
Lahiru

On Tue, May 6, 2014 at 4:35 PM, Saminda Wijeratne <samindaw@gmail.com>wrote:

> This design is just for the Gateways and we probably have things we
> missed. Raman pointed out offline that we are missing some application
> level properties (eg: environment vars) and job level properties. We are
> not thinking of looking at how the backend registry is going to support
> this until we get a decent design for the API Thrift models and functions.
>
>
> On Tue, May 6, 2014 at 10:40 AM, Marlon Pierce <marpierc@iu.edu> wrote:
>
>> Any feedback on this?
>>
>> Marlon
>>
>> On 5/5/14 4:12 PM, Sachith Withana wrote:
>> > Hi all,After an offline discussion, we came up with an initial
>> Application
>> > Catalog design which captures the minimum requirements.
>> >
>> > Any suggestions?
>> > Functionalities for the GatewayAPI Thrift Functions
>> >
>> >    -
>> >
>> >    add // (not required for tutorial but will make life easier for us)
>> >    -
>> >
>> >       String add(application)  //return application id
>> >       -
>> >
>> >       String add(applicationId,deployment) //return deployment id
>> >       -
>> >
>> >    get
>> >    -
>> >
>> >       get(applicationId)  //return application
>> >       -
>> >
>> >       get(applicationId, deploymentId) //return deployment
>> >       -
>> >
>> >    remove //optional
>> >    -
>> >
>> >       remove(applicationId)
>> >       -
>> >
>> >       remove(applicationId, deploymentId)
>> >       -
>> >
>> >    update //optional
>> >    -
>> >
>> >       update(application)
>> >       -
>> >
>> >       update(applicationId, deployment)
>> >       -
>> >
>> >    list
>> >    -
>> >
>> >       String[] list()  // list of application ids
>> >       -
>> >
>> >       String[] list(applicationId) // list of deployment ids
>> >
>> >
>> > API Thrift Data model
>> >
>> > Application (application id)
>> >
>> > Input* : name/type/optional?
>> >
>> > Output* : name/type/optional?/
>> >
>> > Deployment* (deployment id)
>> >
>> >  Host (host id)
>> >
>> >  host IP
>> >
>> >  host job management protocol and properties
>> >
>> >  host data management protocol and properties
>> >
>> >  executable path
>> >
>> >  scratch location
>> >
>> >  job properties (key/value default values)
>> >
>> >  eg: Project Id, CPU count, Node count, Wall time
>> >
>> >  *0 or more
>> >
>> > BOLD : minimum requirement
>> >
>>
>>
>


-- 
System Analyst Programmer
PTI Lab
Indiana University

Mime
View raw message