airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saminda Wijeratne <samin...@gmail.com>
Subject Re: Application Catalog Design Phase-I
Date Tue, 13 May 2014 14:56:19 GMT
I do not know yet whether how thrift supports inheritance but I was
thinking more of the lines of renaming each function to the same name. i.e.
addSSHJobSubmissionProtocol(string, SSHJobSubmissionProtocol)
addGSISSHJobSubmissionProtocol(string, GSISSHJobSubmissionProtocol)
addGlobusJobSubmissionProtocol(string, GlobusJobSubmissionProtocol)
to
addJobSubmissionProtocol(string, SSHJobSubmissionProtocol)
addJobSubmissionProtocol(string, GSISSHJobSubmissionProtocol)
addJobSubmissionProtocol(string, GlobusJobSubmissionProtocol)

I do not know whether passing a generic object type as follows is possible
in thrift,
addJobSubmissionProtocol(string, JobSubmissionProtocolEnum,
JobSubmissionProtocolObject)
I will test it and see.


On Tue, May 13, 2014 at 7:46 AM, Marlon Pierce <marpierc@iu.edu> wrote:

> I vote yes. Otherwise, every time we add new submission protocols, we
> will need to add new API methods.  With 1 method, we would just need to
> add a new protocol type:
>
> addJobSumissionProtocol(..., JobSumissionProtocol.SSH, ...), etc.
>
> Marlon
>
> On 5/13/14 10:41 AM, Saminda Wijeratne wrote:
> > Shall I collapse following 3 functions to 3 functions of
> > addJobSubmissionProtocol(...)
> > with corresponding parameters (overloading)?
> > addSSHJobSubmissionProtocol(string, SSHJobSubmissionProtocol)
> > addGSISSHJobSubmissionProtocol(string, GSISSHJobSubmissionProtocol)
> > addGlobusJobSubmissionProtocol(string, GlobusJobSubmissionProtocol)
> >
> > And of course do the same to the data movement?
> >
> > On Mon, May 12, 2014 at 9:45 PM, Saminda Wijeratne <samindaw@gmail.com
> >wrote:
> >
> >> +1. If no other objections I'll get the thrift skeletons and stubs
> >> generated tomorrow and get the basic implementation for the service done
> >> ASAP.
> >>
> >>
> >> On Mon, May 12, 2014 at 10:27 AM, Suresh Marru <smarru@apache.org>
> wrote:
> >>
> >>> + 1.
> >>>
> >>> Just one cosmetic name change to consider, may be
> >>> the applicationCatalog.thrift should be also explicit and name
> >>> it applicationCatalogAPI.thrift?
> >>>
> >>> Suresh
> >>>
> >>> On May 12, 2014, at 2:49 AM, Saminda Wijeratne <samindaw@gmail.com>
> >>> wrote:
> >>>
> >>> heads up on the name change for the above mentioned thrift file to
> >>> applicationCatalogDataModel.thrift [1].
> >>>
> >>> Also added a new thrift file to define the initial ApplicationCatalog
> >>> related API functions @ [2]. It has,
> >>>
> >>>    1. Functions to manage ComputerResourceDescriptions
> >>>    2. Functions to manage ApplicationInterfaces (for completeness)
> >>>    3. Functions to manage ApplicationDeployments (for completeness)
> >>>
> >>> Thanks,
> >>> Saminda
> >>>
> >>> 1.
> >>>
> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob;f=airavata-api/thrift-interface-descriptions/applicationCatalogDataModel.thrift;h=65314f06de3bc8c3bc31c41c8109d89f4265fcbf;hb=HEAD
> >>> 2.
> >>>
> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob;f=airavata-api/thrift-interface-descriptions/applicationCatalog.thrift;h=19f81e5db1dcadf9d636beab6325dc795d9eab7e;hb=HEAD
> >>>
> >>>
> >>> On Sun, May 11, 2014 at 12:25 PM, Saminda Wijeratne <
> samindaw@gmail.com>wrote:
> >>>
> >>>> Thank you all for your feedback from both the mailing list and google
> >>>> hangout discussions. As for the last weeks google hangout discussion
> we
> >>>> came to the following conclusions regarding phase 1.
> >>>>
> >>>>
> >>>>    - Focus only on thrift data model ComputeResourceDescription[1]
> >>>>    (i.e. hosts)
> >>>>       - we'll re-evaluate the design after the tutorial
> >>>>    - Thrift API functions will be defined for the thrift data model
> >>>>    ComputeResourceDescription and its supporting models in a separate
> thrift
> >>>>    file
> >>>>    - Thrift data models for the rest of the Application Catalog
> >>>>    components will be focussed upon later
> >>>>    - Reuse the existing ServiceDescriptor and ApplicationDescriptor
> >>>>       schema objects for the time-being to fill the missing void in
> the
> >>>>       application catalog.
> >>>>    - A new component called "Application Catalog" will be introduced
> >>>>    along with a CPI for it
> >>>>    - We will be saving ComputeResourceDescription as the way we save
> >>>>    thrift objects. But ApplicationDescriptor and ServiceDescriptor
> will be
> >>>>    saved as blobs with basic id/name metadata fields to allow
> querying.
> >>>>    - Samples and tests which used descriptors to add applications will
> >>>>    use the new CPI/API functions as necessary to add applications to
> the
> >>>>    Application Catalog
> >>>>    - GFac will retrieve applications from the application catalog
> >>>>    retiring the old Airavata API completely. However in order to
> minimize the
> >>>>    changes at GFac side it will transform the application catalog to
> schema
> >>>>    objects service/host/application descriptors.
> >>>>    - We will attempt to complete these changes by 16th May in order
to
> >>>>    start testing and using them by next week.
> >>>>
> >>>>
> >>>> Thanks,
> >>>> Saminda
> >>>>
> >>>> 1.
> >>>>
> https://git-wip-us.apache.org/repos/asf?p=airavata.git;a=blob;f=airavata-api/thrift-interface-descriptions/computeResourceDescription.thrift;h=c025f12ade908fbdd395a1d4c64f811e49f4dd3d;hb=HEAD
> >>>>
> >>>> On Tue, May 6, 2014 at 12:11 PM, Raminder Singh <
> >>>> raminderjsingh@gmail.com> wrote:
> >>>>
> >>>>> Thanks Sachith.  We can have host object independent of application
> as
> >>>>> the same host definition can be used for multiple applications.
> Please add
> >>>>> complete details of host and application properties like Application
> >>>>> type(MPI, OpenMP, MapReduce etc), JobManager (PBS, SLURM,EC2). You
> need to
> >>>>> consider extensibility of the models also as requirements may change
> with
> >>>>> job or monitoring manager details.
> >>>>>
> >>>>> Thanks
> >>>>> Raminder
> >>>>>
> >>>>> On May 5, 2014, at 4:12 PM, Sachith Withana <swsachith@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>> Hi all, After an offline discussion, we came up with an initial
> >>>>> Application Catalog design which captures the minimum requirements.
> >>>>>
> >>>>> Any suggestions?
> >>>>> Functionalities for the Gateway API Thrift Functions
> >>>>>
> >>>>>    - add // (not required for tutorial but will make life easier
for
> >>>>>    us)
> >>>>>    - String add(application)  //return application id
> >>>>>
> >>>>>
> >>>>>    -
> >>>>>       - String add(applicationId,deployment) //return deployment
id
> >>>>>       - get
> >>>>>    - get(applicationId)  //return application
> >>>>>       - get(applicationId, deploymentId) //return deployment
> >>>>>       - remove //optional
> >>>>>    - remove(applicationId)
> >>>>>       - remove(applicationId, deploymentId)
> >>>>>       - update //optional
> >>>>>    - update(application)
> >>>>>       - update(applicationId, deployment)
> >>>>>       - list
> >>>>>    - String[] list()  // list of application ids
> >>>>>       - String[] list(applicationId) // list of deployment ids
> >>>>>
> >>>>>
> >>>>> API Thrift Data model
> >>>>> Application (application id)
> >>>>>  Input* : name/type/optional?
> >>>>>  Output* : name/type/optional?/
> >>>>>  Deployment* (deployment id)
> >>>>>  Host (host id)
> >>>>>  host IP
> >>>>>  host job management protocol and properties
> >>>>>  host data management protocol and properties
> >>>>>  executable path
> >>>>>  scratch location
> >>>>>  job properties (key/value default values)
> >>>>>   eg: Project Id, CPU count, Node count, Wall time
> >>>>>
> >>>>>   *0 or more
> >>>>> BOLD : minimum requirement
> >>>>>
> >>>>> --
> >>>>> Thanks,
> >>>>> Sachith Withana
> >>>>>
> >>>>>
> >>>>>
> >>>
>
>

Mime
View raw message