airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marlon Pierce <marpi...@iu.edu>
Subject Re: Application Catalog Design Phase-I
Date Tue, 13 May 2014 14:46:33 GMT
I vote yes. Otherwise, every time we add new submission protocols, we
will need to add new API methods.  With 1 method, we would just need to
add a new protocol type:

addJobSumissionProtocol(..., JobSumissionProtocol.SSH, ...), etc.

Marlon

On 5/13/14 10:41 AM, Saminda Wijeratne wrote:
> Shall I collapse following 3 functions to 3 functions of
> addJobSubmissionProtocol(...)
> with corresponding parameters (overloading)?
> addSSHJobSubmissionProtocol(string, SSHJobSubmissionProtocol)
> addGSISSHJobSubmissionProtocol(string, GSISSHJobSubmissionProtocol)
> addGlobusJobSubmissionProtocol(string, GlobusJobSubmissionProtocol)
>
> And of course do the same to the data movement?
>
> On Mon, May 12, 2014 at 9:45 PM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>
>> +1. If no other objections I'll get the thrift skeletons and stubs
>> generated tomorrow and get the basic implementation for the service done
>> ASAP.
>>
>>
>> On Mon, May 12, 2014 at 10:27 AM, Suresh Marru <smarru@apache.org> wrote:
>>
>>> + 1.
>>>
>>> Just one cosmetic name change to consider, may be
>>> the applicationCatalog.thrift should be also explicit and name
>>> it applicationCatalogAPI.thrift?
>>>
>>> Suresh
>>>
>>> On May 12, 2014, at 2:49 AM, Saminda Wijeratne <samindaw@gmail.com>
>>> wrote:
>>>
>>> heads up on the name change for the above mentioned thrift file to
>>> applicationCatalogDataModel.thrift [1].
>>>
>>> Also added a new thrift file to define the initial ApplicationCatalog
>>> related API functions @ [2]. It has,
>>>
>>>    1. Functions to manage ComputerResourceDescriptions
>>>    2. Functions to manage ApplicationInterfaces (for completeness)
>>>    3. Functions to manage ApplicationDeployments (for completeness)
>>>
>>> Thanks,
>>> Saminda
>>>
>>> 1.
>>> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob;f=airavata-api/thrift-interface-descriptions/applicationCatalogDataModel.thrift;h=65314f06de3bc8c3bc31c41c8109d89f4265fcbf;hb=HEAD
>>> 2.
>>> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob;f=airavata-api/thrift-interface-descriptions/applicationCatalog.thrift;h=19f81e5db1dcadf9d636beab6325dc795d9eab7e;hb=HEAD
>>>
>>>
>>> On Sun, May 11, 2014 at 12:25 PM, Saminda Wijeratne <samindaw@gmail.com>wrote:
>>>
>>>> Thank you all for your feedback from both the mailing list and google
>>>> hangout discussions. As for the last weeks google hangout discussion we
>>>> came to the following conclusions regarding phase 1.
>>>>
>>>>
>>>>    - Focus only on thrift data model ComputeResourceDescription[1]
>>>>    (i.e. hosts)
>>>>       - we'll re-evaluate the design after the tutorial
>>>>    - Thrift API functions will be defined for the thrift data model
>>>>    ComputeResourceDescription and its supporting models in a separate thrift
>>>>    file
>>>>    - Thrift data models for the rest of the Application Catalog
>>>>    components will be focussed upon later
>>>>    - Reuse the existing ServiceDescriptor and ApplicationDescriptor
>>>>       schema objects for the time-being to fill the missing void in the
>>>>       application catalog.
>>>>    - A new component called "Application Catalog" will be introduced
>>>>    along with a CPI for it
>>>>    - We will be saving ComputeResourceDescription as the way we save
>>>>    thrift objects. But ApplicationDescriptor and ServiceDescriptor will be
>>>>    saved as blobs with basic id/name metadata fields to allow querying.
>>>>    - Samples and tests which used descriptors to add applications will
>>>>    use the new CPI/API functions as necessary to add applications to the
>>>>    Application Catalog
>>>>    - GFac will retrieve applications from the application catalog
>>>>    retiring the old Airavata API completely. However in order to minimize
the
>>>>    changes at GFac side it will transform the application catalog to schema
>>>>    objects service/host/application descriptors.
>>>>    - We will attempt to complete these changes by 16th May in order to
>>>>    start testing and using them by next week.
>>>>
>>>>
>>>> Thanks,
>>>> Saminda
>>>>
>>>> 1.
>>>> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob;f=airavata-api/thrift-interface-descriptions/computeResourceDescription.thrift;h=c025f12ade908fbdd395a1d4c64f811e49f4dd3d;hb=HEAD
>>>>
>>>> On Tue, May 6, 2014 at 12:11 PM, Raminder Singh <
>>>> raminderjsingh@gmail.com> wrote:
>>>>
>>>>> Thanks Sachith.  We can have host object independent of application as
>>>>> the same host definition can be used for multiple applications. Please
add
>>>>> complete details of host and application properties like Application
>>>>> type(MPI, OpenMP, MapReduce etc), JobManager (PBS, SLURM,EC2). You need
to
>>>>> consider extensibility of the models also as requirements may change
with
>>>>> job or monitoring manager details.
>>>>>
>>>>> Thanks
>>>>> Raminder
>>>>>
>>>>> On May 5, 2014, at 4:12 PM, Sachith Withana <swsachith@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi all, After an offline discussion, we came up with an initial
>>>>> Application Catalog design which captures the minimum requirements.
>>>>>
>>>>> Any suggestions?
>>>>> Functionalities for the Gateway API Thrift Functions
>>>>>
>>>>>    - add // (not required for tutorial but will make life easier for
>>>>>    us)
>>>>>    - String add(application)  //return application id
>>>>>
>>>>>
>>>>>    -
>>>>>       - String add(applicationId,deployment) //return deployment id
>>>>>       - get
>>>>>    - get(applicationId)  //return application
>>>>>       - get(applicationId, deploymentId) //return deployment
>>>>>       - remove //optional
>>>>>    - remove(applicationId)
>>>>>       - remove(applicationId, deploymentId)
>>>>>       - update //optional
>>>>>    - update(application)
>>>>>       - update(applicationId, deployment)
>>>>>       - list
>>>>>    - String[] list()  // list of application ids
>>>>>       - String[] list(applicationId) // list of deployment ids
>>>>>
>>>>>
>>>>> API Thrift Data model
>>>>> Application (application id)
>>>>>  Input* : name/type/optional?
>>>>>  Output* : name/type/optional?/
>>>>>  Deployment* (deployment id)
>>>>>  Host (host id)
>>>>>  host IP
>>>>>  host job management protocol and properties
>>>>>  host data management protocol and properties
>>>>>  executable path
>>>>>  scratch location
>>>>>  job properties (key/value default values)
>>>>>   eg: Project Id, CPU count, Node count, Wall time
>>>>>
>>>>>   *0 or more
>>>>> BOLD : minimum requirement
>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Sachith Withana
>>>>>
>>>>>
>>>>>
>>>


Mime
View raw message