airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saminda Wijeratne <samin...@gmail.com>
Subject Re: Application Catalog Design Phase-I
Date Tue, 13 May 2014 14:41:32 GMT
Shall I collapse following 3 functions to 3 functions of
addJobSubmissionProtocol(...)
with corresponding parameters (overloading)?
addSSHJobSubmissionProtocol(string, SSHJobSubmissionProtocol)
addGSISSHJobSubmissionProtocol(string, GSISSHJobSubmissionProtocol)
addGlobusJobSubmissionProtocol(string, GlobusJobSubmissionProtocol)

And of course do the same to the data movement?

On Mon, May 12, 2014 at 9:45 PM, Saminda Wijeratne <samindaw@gmail.com>wrote:

> +1. If no other objections I'll get the thrift skeletons and stubs
> generated tomorrow and get the basic implementation for the service done
> ASAP.
>
>
> On Mon, May 12, 2014 at 10:27 AM, Suresh Marru <smarru@apache.org> wrote:
>
>> + 1.
>>
>> Just one cosmetic name change to consider, may be
>> the applicationCatalog.thrift should be also explicit and name
>> it applicationCatalogAPI.thrift?
>>
>> Suresh
>>
>> On May 12, 2014, at 2:49 AM, Saminda Wijeratne <samindaw@gmail.com>
>> wrote:
>>
>> heads up on the name change for the above mentioned thrift file to
>> applicationCatalogDataModel.thrift [1].
>>
>> Also added a new thrift file to define the initial ApplicationCatalog
>> related API functions @ [2]. It has,
>>
>>    1. Functions to manage ComputerResourceDescriptions
>>    2. Functions to manage ApplicationInterfaces (for completeness)
>>    3. Functions to manage ApplicationDeployments (for completeness)
>>
>> Thanks,
>> Saminda
>>
>> 1.
>> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob;f=airavata-api/thrift-interface-descriptions/applicationCatalogDataModel.thrift;h=65314f06de3bc8c3bc31c41c8109d89f4265fcbf;hb=HEAD
>> 2.
>> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob;f=airavata-api/thrift-interface-descriptions/applicationCatalog.thrift;h=19f81e5db1dcadf9d636beab6325dc795d9eab7e;hb=HEAD
>>
>>
>> On Sun, May 11, 2014 at 12:25 PM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>>
>>> Thank you all for your feedback from both the mailing list and google
>>> hangout discussions. As for the last weeks google hangout discussion we
>>> came to the following conclusions regarding phase 1.
>>>
>>>
>>>    - Focus only on thrift data model ComputeResourceDescription[1]
>>>    (i.e. hosts)
>>>       - we'll re-evaluate the design after the tutorial
>>>    - Thrift API functions will be defined for the thrift data model
>>>    ComputeResourceDescription and its supporting models in a separate thrift
>>>    file
>>>    - Thrift data models for the rest of the Application Catalog
>>>    components will be focussed upon later
>>>    - Reuse the existing ServiceDescriptor and ApplicationDescriptor
>>>       schema objects for the time-being to fill the missing void in the
>>>       application catalog.
>>>    - A new component called "Application Catalog" will be introduced
>>>    along with a CPI for it
>>>    - We will be saving ComputeResourceDescription as the way we save
>>>    thrift objects. But ApplicationDescriptor and ServiceDescriptor will be
>>>    saved as blobs with basic id/name metadata fields to allow querying.
>>>    - Samples and tests which used descriptors to add applications will
>>>    use the new CPI/API functions as necessary to add applications to the
>>>    Application Catalog
>>>    - GFac will retrieve applications from the application catalog
>>>    retiring the old Airavata API completely. However in order to minimize the
>>>    changes at GFac side it will transform the application catalog to schema
>>>    objects service/host/application descriptors.
>>>    - We will attempt to complete these changes by 16th May in order to
>>>    start testing and using them by next week.
>>>
>>>
>>> Thanks,
>>> Saminda
>>>
>>> 1.
>>> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob;f=airavata-api/thrift-interface-descriptions/computeResourceDescription.thrift;h=c025f12ade908fbdd395a1d4c64f811e49f4dd3d;hb=HEAD
>>>
>>> On Tue, May 6, 2014 at 12:11 PM, Raminder Singh <
>>> raminderjsingh@gmail.com> wrote:
>>>
>>>> Thanks Sachith.  We can have host object independent of application as
>>>> the same host definition can be used for multiple applications. Please add
>>>> complete details of host and application properties like Application
>>>> type(MPI, OpenMP, MapReduce etc), JobManager (PBS, SLURM,EC2). You need to
>>>> consider extensibility of the models also as requirements may change with
>>>> job or monitoring manager details.
>>>>
>>>> Thanks
>>>> Raminder
>>>>
>>>> On May 5, 2014, at 4:12 PM, Sachith Withana <swsachith@gmail.com>
>>>> wrote:
>>>>
>>>> Hi all, After an offline discussion, we came up with an initial
>>>> Application Catalog design which captures the minimum requirements.
>>>>
>>>> Any suggestions?
>>>> Functionalities for the Gateway API Thrift Functions
>>>>
>>>>    - add // (not required for tutorial but will make life easier for
>>>>    us)
>>>>    - String add(application)  //return application id
>>>>
>>>>
>>>>    -
>>>>       - String add(applicationId,deployment) //return deployment id
>>>>       - get
>>>>    - get(applicationId)  //return application
>>>>       - get(applicationId, deploymentId) //return deployment
>>>>       - remove //optional
>>>>    - remove(applicationId)
>>>>       - remove(applicationId, deploymentId)
>>>>       - update //optional
>>>>    - update(application)
>>>>       - update(applicationId, deployment)
>>>>       - list
>>>>    - String[] list()  // list of application ids
>>>>       - String[] list(applicationId) // list of deployment ids
>>>>
>>>>
>>>> API Thrift Data model
>>>> Application (application id)
>>>>  Input* : name/type/optional?
>>>>  Output* : name/type/optional?/
>>>>  Deployment* (deployment id)
>>>>  Host (host id)
>>>>  host IP
>>>>  host job management protocol and properties
>>>>  host data management protocol and properties
>>>>  executable path
>>>>  scratch location
>>>>  job properties (key/value default values)
>>>>   eg: Project Id, CPU count, Node count, Wall time
>>>>
>>>>   *0 or more
>>>> BOLD : minimum requirement
>>>>
>>>> --
>>>> Thanks,
>>>> Sachith Withana
>>>>
>>>>
>>>>
>>>
>>
>>
>

Mime
View raw message