; Wed, 16 Dec 2015 16:25:39 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1720388 [3/5] - /airavata/site/trunk/content/documentation/api-docs/0.16/ Date: Wed, 16 Dec 2015 16:25:38 -0000 To: commits@airavata.apache.org From: smarru@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20151216162539.162BF3A158A@svn01-us-west.apache.org> Added: airavata/site/trunk/content/documentation/api-docs/0.16/airavata_commons.html URL: http://svn.apache.org/viewvc/airavata/site/trunk/content/documentation/api-docs/0.16/airavata_commons.html?rev=1720388&view=auto ============================================================================== --- airavata/site/trunk/content/documentation/api-docs/0.16/airavata_commons.html (added) +++ airavata/site/trunk/content/documentation/api-docs/0.16/airavata_commons.html Wed Dec 16 16:25:38 2015 @@ -0,0 +1,44 @@ + + + + + +Thrift module: airavata_commons +

Thrift module: airavata_commons

+ + + + + +

Module	Services	Data types	Constants
airavata_commons		ErrorModel +ValidationResults +ValidatorResult +	`DEFAULT_ID` +

Constants

+ +

Constant	Type	Value
`DEFAULT_ID`	`string`	`"DO_NOT_SET_AT_CLIENTS"`

Data structures

Struct: ErrorModel

+ + + + + + + +

Key	Field	Type	Requiredness	Default value
1	errorId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	creationTime	`i64`	optional
3	actualErrorMessage	`string`	optional
4	userFriendlyMessage	`string`	optional
5	transientOrPersistent	`bool`	optional	`false`
6	rootCauseErrorIdList	`list<string>`	optional

Struct: ValidatorResult

+ + + +

Key	Field	Type	Description	Requiredness	Default value
1	result	`bool`		required
2	errorDetails	`string`		optional

This data structure can be used to store the validation results +captured during validation step and during the launchExperiment +operation it can be easilly checked to see the errors occured +during the experiment launch operation + +

Struct: ValidationResults

+ + + +

Key	Field	Type	Description	Requiredness	Default value
1	validationState	`bool`		required
2	validationResultList	`list<ValidatorResult>`		required

Added: airavata/site/trunk/content/documentation/api-docs/0.16/airavata_data_models.html URL: http://svn.apache.org/viewvc/airavata/site/trunk/content/documentation/api-docs/0.16/airavata_data_models.html?rev=1720388&view=auto ============================================================================== --- airavata/site/trunk/content/documentation/api-docs/0.16/airavata_data_models.html (added) +++ airavata/site/trunk/content/documentation/api-docs/0.16/airavata_data_models.html Wed Dec 16 16:25:38 2015 @@ -0,0 +1,16 @@ + + + + + +Thrift module: airavata_data_models +

Thrift module: airavata_data_models

+ + + + + +

Module	Services	Data types	Constants
airavata_data_models

Added: airavata/site/trunk/content/documentation/api-docs/0.16/airavata_errors.html URL: http://svn.apache.org/viewvc/airavata/site/trunk/content/documentation/api-docs/0.16/airavata_errors.html?rev=1720388&view=auto ============================================================================== --- airavata/site/trunk/content/documentation/api-docs/0.16/airavata_errors.html (added) +++ airavata/site/trunk/content/documentation/api-docs/0.16/airavata_errors.html Wed Dec 16 16:25:38 2015 @@ -0,0 +1,143 @@ + + + + + +Thrift module: airavata_errors +

Thrift module: airavata_errors

+ + + + + +

Module	Services	Data types	Constants
airavata_errors		AiravataClientException +AiravataErrorType +AiravataSystemException +AuthenticationException +AuthorizationException +ExperimentNotFoundException +InvalidRequestException +LaunchValidationException +ProjectNotFoundException +TimedOutException +ValidationResults +ValidatorResult +

Enumerations

Enumeration: AiravataErrorType

+A list of Airavata API Error Message Types + + UNKNOWN: No information available about the error + + PERMISSION_DENIED: Not permitted to perform action + + INTERNAL_ERROR: Unexpected problem with the service + + AUTHENTICATION_FAILURE: The client failed to authenticate. + + INVALID_AUTHORIZATION: Security Token and/or Username and/or password is incorrect + + AUTHORIZATION_EXPIRED: Authentication token expired + + UNKNOWN_GATEWAY_ID: The gateway is not registered with Airavata. + + UNSUPPORTED_OPERATION: Operation denied because it is currently unsupported. +

+ + + + + + + + +

`UNKNOWN`	`0`	+
`PERMISSION_DENIED`	`1`	+
`INTERNAL_ERROR`	`2`	+
`AUTHENTICATION_FAILURE`	`3`	+
`INVALID_AUTHORIZATION`	`4`	+
`AUTHORIZATION_EXPIRED`	`5`	+
`UNKNOWN_GATEWAY_ID`	`6`	+
`UNSUPPORTED_OPERATION`	`7`	+

Data structures

Exception: ExperimentNotFoundException

+ + +

Key	Field	Type	Description	Requiredness	Default value
1	message	`string`		required

This exception is thrown when a client asks to perform an operation on an experiment that does not exist. + +identifier: A description of the experiment that was not found on the server. + +key: The value passed from the client in the identifier, which was not found. +

Exception: ProjectNotFoundException

+ + +

Key	Field	Type	Description	Requiredness	Default value
1	message	`string`		required

1: optional string identifier, +2: optional string key + +

Exception: InvalidRequestException

+ + +

Key	Field	Type	Description	Requiredness	Default value
1	message	`string`		required

This exception is thrown for invalid requests that occur from any reasons like required input parameters are missing, + or a parameter is malformed. + + message: contains the associated error message. +

Exception: TimedOutException

+ +

Key	Field	Type	Description	Requiredness	Default value

This exception is thrown when RPC timeout gets exceeded. +

Exception: AuthenticationException

+ + +

Key	Field	Type	Description	Requiredness	Default value
1	message	`string`		required

This exception is thrown for invalid sshKeyAuthentication requests. + + message: contains the cause of the authorization failure. +

Exception: AuthorizationException

+ + +

Key	Field	Type	Description	Requiredness	Default value
1	message	`string`		required

This exception is thrown for invalid authorization requests such user does not have acces to an aplication or resource. + + message: contains the authorization failure message +

Exception: AiravataClientException

+ + + +

Key	Field	Type	Description	Requiredness	Default value
1	airavataErrorType	`AiravataErrorType`		required
2	parameter	`string`		optional

This exception is thrown by Airavata Services when a call fails as a result of +a problem that a client may be able to resolve. For example, if the user +attempts to execute an application on a resource gateway does not have access to. + +This exception would not be used for internal system errors that do not +reflect user actions, but rather reflect a problem within the service that +the client cannot resolve. + +airavataErrorType: The message type indicating the error that occurred. + must be one of the values of AiravataErrorType. + +parameter: If the error applied to a particular input parameter, this will + indicate which parameter. +

Struct: ValidatorResult

+ + + +

Key	Field	Type	Description	Requiredness	Default value
1	result	`bool`		required
2	errorDetails	`string`		optional

Struct: ValidationResults

+ + + +

Key	Field	Type	Description	Requiredness	Default value
1	validationState	`bool`		required
2	validationResultList	`list<ValidatorResult>`		required

Exception: LaunchValidationException

+ + + +

Key	Field	Type	Description	Requiredness	Default value
1	validationResult	`ValidationResults`		required
2	errorMessage	`string`		optional

Exception: AiravataSystemException

+ + + +

Key	Field	Type	Description	Requiredness	Default value
1	airavataErrorType	`AiravataErrorType`		required
2	message	`string`		optional

This exception is thrown by Airavata Services when a call fails as a result of +a problem in the service that could not be changed through client's action. + +airavataErrorType: The message type indicating the error that occurred. + must be one of the values of AiravataErrorType. + +message: This may contain additional information about the error + +

Added: airavata/site/trunk/content/documentation/api-docs/0.16/application_deployment_model.html URL: http://svn.apache.org/viewvc/airavata/site/trunk/content/documentation/api-docs/0.16/application_deployment_model.html?rev=1720388&view=auto ============================================================================== --- airavata/site/trunk/content/documentation/api-docs/0.16/application_deployment_model.html (added) +++ airavata/site/trunk/content/documentation/api-docs/0.16/application_deployment_model.html Wed Dec 16 16:25:38 2015 @@ -0,0 +1,138 @@ + + + + + +Thrift module: application_deployment_model +

Thrift module: application_deployment_model

+This file describes the definitions of the Data Structures related to deployment of Application on + computational resources. + +
+ + + + +

Module	Services	Data types	Constants
application_deployment_model		ApplicationDeploymentDescription +ApplicationModule +ApplicationParallelismType +CommandObject +SetEnvPaths +

Enumerations

Enumeration: ApplicationParallelismType

+Enumeration of application parallelism supported by Airavata + +SERIAL: + Single processor applications without any parallelization. + +MPI: + Messaging Passing Interface. + +OPENMP: + Shared Memory Implementtaion. + +OPENMP_MPI: + Hybrid Applications. + +

+ + + + + + +

`SERIAL`	`0`	+
`MPI`	`1`	+
`OPENMP`	`2`	+
`OPENMP_MPI`	`3`	+
`CCM`	`4`	+
`CRAY_MPI`	`5`	+

Data structures

Struct: SetEnvPaths

+ + + + +

Key	Field	Type	Requiredness
1	name	`string`	required
2	value	`string`	required
3	envPathOrder	`i32`	optional

Key Value pairs to be used to set environments + +name: + Name of the environment variable such as PATH, LD_LIBRARY_PATH, NETCDF_HOME. + +value: + Value of the environment variable to set + +envPathOrder: + The order of the setting of the env variables when there are multiple env variables +

Struct: CommandObject

+ + + +

Key	Field	Type	Description	Requiredness	Default value
1	command	`string`		required
2	commandOrder	`i32`		optional

Job commands to be used in Pre Job, Post Job and Module Load Commands + +command: + The actual command in string format + +commandOrder: + Order of the command in the multiple command situation +

Struct: ApplicationModule

+ + + + + +

Key	Field	Type	Requiredness	Default value
1	appModuleId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	appModuleName	`string`	required
3	appModuleVersion	`string`	optional
4	appModuleDescription	`string`	optional

Application Module Information. A module has to be registered before registering a deployment. + +appModuleId: Airavata Internal Unique Job ID. This is set by the registry. + +appModuleName: + Name of the application module. + +appModuleVersion: + Version of the application. + +appModuleDescription: + Descriprion of the Module + +

Struct: ApplicationDeploymentDescription

+ + + + + + + + + + + + + +

Key	Field	Type	Requiredness	Default value
1	appDeploymentId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	appModuleId	`string`	required
3	computeHostId	`string`	required
4	executablePath	`string`	required
5	parallelism	`ApplicationParallelismType`	required	`ApplicationParallelismType.SERIAL`
6	appDeploymentDescription	`string`	optional
7	moduleLoadCmds	`list<CommandObject>`	optional
8	libPrependPaths	`list<SetEnvPaths>`	optional
9	libAppendPaths	`list<SetEnvPaths>`	optional
10	setEnvironment	`list<SetEnvPaths>`	optional
11	preJobCommands	`list<CommandObject>`	optional
12	postJobCommands	`list<CommandObject>`	optional

Application Deployment Description + +appDeploymentId: Airavata Internal Unique Job ID. This is set by the registry. + +appModuleName: + Application Module Name. This has to be precise describing the binary. + +computeHostId: + This ID maps application deployment to a particular resource previously described within Airavata. + Example: Stampede is first registered and refered when registering WRF. + +moduleLoadCmd: + Command string to load modules. This will be placed in the job submisison + Ex: module load amber + +libPrependPaths: + prepend to a path variable the value + +libAppendPaths: + append to a path variable the value + +setEnvironment: + assigns to the environment variable "NAME" the value + +

Added: airavata/site/trunk/content/documentation/api-docs/0.16/application_interface_model.html URL: http://svn.apache.org/viewvc/airavata/site/trunk/content/documentation/api-docs/0.16/application_interface_model.html?rev=1720388&view=auto ============================================================================== --- airavata/site/trunk/content/documentation/api-docs/0.16/application_interface_model.html (added) +++ airavata/site/trunk/content/documentation/api-docs/0.16/application_interface_model.html Wed Dec 16 16:25:38 2015 @@ -0,0 +1,40 @@ + + + + + +Thrift module: application_interface_model +

Thrift module: application_interface_model

+This file describes the definitions of the Data Structures of Application interfaces. These interfaces are mapped + to application mapping on various resources. + +
+ + + + +

Module	Services	Data types	Constants
application_interface_model		ApplicationInterfaceDescription +

Data structures

Struct: ApplicationInterfaceDescription

+ + + + + + + +

Key	Field	Type	Requiredness	Default value
1	applicationInterfaceId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	applicationName	`string`	required
3	applicationDescription	`string`	optional
4	applicationModules	`list<string>`	optional
5	applicationInputs	`list<application_io_models.InputDataObjectType>`	optional
6	applicationOutputs	`list<application_io_models.OutputDataObjectType>`	optional

Application Interface Description + +applicationModules: + Associate all application modules with versions which interface is applicable to. + +applicationInputs: + Inputs to be passed to the application + +applicationOutputs: + Outputs generated from the application + +

Added: airavata/site/trunk/content/documentation/api-docs/0.16/application_io_models.html URL: http://svn.apache.org/viewvc/airavata/site/trunk/content/documentation/api-docs/0.16/application_io_models.html?rev=1720388&view=auto ============================================================================== --- airavata/site/trunk/content/documentation/api-docs/0.16/application_io_models.html (added) +++ airavata/site/trunk/content/documentation/api-docs/0.16/application_io_models.html Wed Dec 16 16:25:38 2015 @@ -0,0 +1,111 @@ + + + + + +Thrift module: application_io_models +

Thrift module: application_io_models

+ + + + + +

Module	Services	Data types	Constants
application_io_models		DataType +InputDataObjectType +OutputDataObjectType +

Enumerations

Enumeration: DataType

+Data Types supported in Airavata. The primitive data types + +

+ + + + + + +

`STRING`	`0`	+
`INTEGER`	`1`	+
`FLOAT`	`2`	+
`URI`	`3`	+
`STDOUT`	`4`	+
`STDERR`	`5`	+

Data structures

Struct: InputDataObjectType

+ + + + + + + + + + + + +

Key	Field	Type	Requiredness
1	name	`string`	required
2	value	`string`	optional
3	type	`DataType`	optional
4	applicationArgument	`string`	optional
5	standardInput	`bool`	optional
6	userFriendlyDescription	`string`	optional
7	metaData	`string`	optional
8	inputOrder	`i32`	optional
9	isRequired	`bool`	optional
10	requiredToAddedToCommandLine	`bool`	optional
11	dataStaged	`bool`	optional

Application Inputs. The paramters describe how inputs are passed to the application. + +name: + Name of the parameter. + +value: + Value of the parameter. A default value could be set during registration. + +type: + Data type of the parameter + +applicationArguement: + The argument flag sent to the application. Such as -p pressure. + +standardInput: + When this value is set, the parameter is sent as standard input rather than a parameter. + Typically this is passed using redirection operator ">". + +userFriendlyDescription: + Description to be displayed at the user interface. + +metaData: + Any metadat. This is typically ignore by Airavata and is used by gateways for application configuration. + +

Struct: OutputDataObjectType

+ + + + + + + + + + + +

Key	Field	Type	Requiredness
1	name	`string`	required
2	value	`string`	optional
3	type	`DataType`	optional
4	applicationArgument	`string`	optional
5	isRequired	`bool`	optional
6	requiredToAddedToCommandLine	`bool`	optional
7	dataMovement	`bool`	optional
8	location	`string`	optional
9	searchQuery	`string`	optional
10	outputStreaming	`bool`	optional

Application Outputs. The paramters describe how outputs generated by the application. + +name: + Name of the parameter. + +value: + Value of the parameter. + +type: + Data type of the parameter + +applicationArguement: + The argument flag sent to the application. Such as -p pressure. + +standardInput: + When this value is set, the parameter is sent as standard input rather than a parameter. + Typically this is passed using redirection operator ">". + +userFriendlyDescription: + Description to be displayed at the user interface. + +metaData: + Any metadat. This is typically ignore by Airavata and is used by gateways for application configuration. + +

Added: airavata/site/trunk/content/documentation/api-docs/0.16/compute_resource_model.html URL: http://svn.apache.org/viewvc/airavata/site/trunk/content/documentation/api-docs/0.16/compute_resource_model.html?rev=1720388&view=auto ============================================================================== --- airavata/site/trunk/content/documentation/api-docs/0.16/compute_resource_model.html (added) +++ airavata/site/trunk/content/documentation/api-docs/0.16/compute_resource_model.html Wed Dec 16 16:25:38 2015 @@ -0,0 +1,347 @@ + + + + + +Thrift module: compute_resource_model +

Thrift module: compute_resource_model

+ + + + + +

Module	Services	Data types	Constants
compute_resource_model		BatchQueue +CloudJobSubmission +ComputeResourceDescription +DMType +FileSystems +GlobusJobSubmission +JobManagerCommand +JobSubmissionInterface +JobSubmissionProtocol +LOCALSubmission +MonitorMode +ProviderName +ResourceJobManager +ResourceJobManagerType +SSHJobSubmission +UnicoreJobSubmission +

Enumerations

Enumeration: ResourceJobManagerType

+* Enumeration of local resource job manager types supported by Airavata +* +* FORK: +* Forking of commands without any job manager +* +* PBS: +* Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine. +* +* SLURM: +* The Simple Linux Utility for Resource Management is a open source workload manager. + * + * UGE: + * Univa Grid Engine, a variation of PBS implementation. + * + * LSF: + * IBM Platform Load Sharing Facility is dominantly installed on IBM clusters. +* +

+ + + + + +

`FORK`	`0`	+
`PBS`	`1`	+
`SLURM`	`2`	+
`LSF`	`3`	+
`UGE`	`4`	+

Enumeration: JobManagerCommand

+Enumeration of resource job manager commands + +SUBMISSION: + Ex: qsub, sbatch + +JOBMONITORING: + Ex: qstat, squeue + +DELETION: + Ex: qdel, scancel + +CHECK_JOB: + Detailed Status about the Job. Ex: checkjob + +SHOW_QUEUE: + List of Queued Job by the schedular. Ex: showq + +SHOW_RESERVATION: + List all reservations. Ex:showres, show_res + +SHOW_START: + Display the start time of the specified job. Ex: showstart + +

+ + + + + + + +

`SUBMISSION`	`0`	+
`JOB_MONITORING`	`1`	+
`DELETION`	`2`	+
`CHECK_JOB`	`3`	+
`SHOW_QUEUE`	`4`	+
`SHOW_RESERVATION`	`5`	+
`SHOW_START`	`6`	+

Enumeration: FileSystems

+Enumeration of File Systems on the resource + +FORK: + Forking of commands without any job manager + +PBS: + Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine. + +UGE: + Univa Grid Engine, a variation of PBS implementation. + +SLURM: + The Simple Linux Utility for Resource Management is a open source workload manager. + +

+ + + + + +

`HOME`	`0`	+
`WORK`	`1`	+
`LOCALTMP`	`2`	+
`SCRATCH`	`3`	+
`ARCHIVE`	`4`	+

Enumeration: JobSubmissionProtocol

+Enumeration of Airavata supported Job Submission Mechanisms for High Performance Computing Clusters. + +SSH: + Execute remote job submission commands using via secure shell protocol. + +GRAM: + Execute remote jobs via Globus GRAM service. + +UNICORE: + Execute remote jobs via Unicore services + +

+ + + + + + + +

`LOCAL`	`0`	+
`SSH`	`1`	+
`GLOBUS`	`2`	+
`UNICORE`	`3`	+
`CLOUD`	`4`	+
`SSH_FORK`	`5`	+
`LOCAL_FORK`	`6`	+

Enumeration: MonitorMode

+Monitoring modes + +POLL_JOB_MANAGER: +GFac need to pull job status changes. + +XSEDE_AMQP_SUBSCRIBE: +Server will publish job status changes to amqp servert. + + +

+ + + + +

`POLL_JOB_MANAGER`	`0`	+
`JOB_EMAIL_NOTIFICATION_MONITOR`	`1`	+
`XSEDE_AMQP_SUBSCRIBE`	`2`	+
`FORK`	`3`	+

Enumeration: DMType

+
+ + +

`COMPUTE_RESOURCE`	`0`	+
`STORAGE_RESOURCE`	`1`	+

Enumeration: ProviderName

+Provider name + +

+ + + +

`EC2`	`0`	+
`AWSEC2`	`1`	+
`RACKSPACE`	`2`	+

Data structures

Struct: ResourceJobManager

+ + + + + + +

Key	Field	Type	Requiredness	Default value
1	resourceJobManagerId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	resourceJobManagerType	`ResourceJobManagerType`	required
3	pushMonitoringEndpoint	`string`	optional
4	jobManagerBinPath	`string`	optional
5	jobManagerCommands	`map<JobManagerCommand, string>`	optional

Resource Job Manager Information + +resourceJobManagerType: + A typical HPC cluster has a single Job Manager to manage the resources. + +pushMonitoringEndpoint: + If the job manager pushes out state changes to a database or bus, specify the service endpoint. + Ex: Moab Web Service, Moab MongoDB URL, AMQP (GLUE2) Broker + +jobManagerBinPath: + Path to the Job Manager Installation Binary directory. + +jobManagerCommands: + An enumeration of commonly used manager commands. + +

Struct: BatchQueue

+ + + + + + + + +

Key	Field	Type	Requiredness
1	queueName	`string`	required
2	queueDescription	`string`	optional
3	maxRunTime	`i32`	optional
4	maxNodes	`i32`	optional
5	maxProcessors	`i32`	optional
6	maxJobsInQueue	`i32`	optional
7	maxMemory	`i32`	optional

Batch Queue Information on SuperComputers + +maxRunTime: + Maximum allowed run time in hours. +

Struct: LOCALSubmission

+ + + + +

Key	Field	Type	Requiredness	Default value
1	jobSubmissionInterfaceId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	securityProtocol	`data_movement_models.SecurityProtocol`	required
3	resourceJobManager	`ResourceJobManager`	required

Locally Fork Jobs as OS processes + +alternativeSSHHostName: + If the login to ssh is different than the hostname itself, specify it here + +sshPort: + If a non-default port needs to used, specify it. +

Struct: SSHJobSubmission

+ + + + + + + + +

Key	Field	Type	Requiredness	Default value
1	jobSubmissionInterfaceId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	securityProtocol	`data_movement_models.SecurityProtocol`	required
3	resourceJobManager	`ResourceJobManager`	required
4	alternativeSSHHostName	`string`	optional
5	sshPort	`i32`	optional	`22`
6	monitorMode	`MonitorMode`	optional
7	batchQueueEmailSenders	`list<string>`	optional

Authenticate using Secured Shell + +alternativeSSHHostName: + If the login to ssh is different than the hostname itself, specify it here + +sshPort: + If a non-default port needs to used, specify it. + +batchQueueEmailSenders: + If a resource always sends the monitoring from a specific address, specify the + full email address. If a resource sends emails from multiple addresses ( + example: based on the submitted login node) then use the wildchar * to indicate + the same. Example: *@*.example.com or *@example.com + +

Struct: GlobusJobSubmission

+ + + + +

Key	Field	Type	Requiredness	Default value
1	jobSubmissionInterfaceId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	securityProtocol	`data_movement_models.SecurityProtocol`	required
3	globusGateKeeperEndPoint	`list<string>`	optional

Struct: UnicoreJobSubmission

+ + + + +

Key	Field	Type	Requiredness	Default value
1	jobSubmissionInterfaceId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	securityProtocol	`data_movement_models.SecurityProtocol`	required
3	unicoreEndPointURL	`string`	required

Unicore Job Submission + +unicoreEndPointURL: + unicoreGateway End Point. The provider will query this service to fetch required service end points. +authenticationMode + The authenticationMode defines the way certificate is fetched. +

Struct: CloudJobSubmission

+ + + + + + + +

Key	Field	Type	Requiredness	Default value
1	jobSubmissionInterfaceId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	securityProtocol	`data_movement_models.SecurityProtocol`	required
3	nodeId	`string`	required
4	executableType	`string`	required
5	providerName	`ProviderName`	required
6	userAccountName	`string`	required

Cloud Job Submission + + +

Struct: JobSubmissionInterface

+ + + + +

Key	Field	Type	Requiredness	Default value
1	jobSubmissionInterfaceId	`string`	required
2	jobSubmissionProtocol	`JobSubmissionProtocol`	required
3	priorityOrder	`i32`	required	`0`

Job Submission Interfaces + +jobSubmissionInterfaceId: The Job Submission Interface has to be previously registered and referenced here. + +priorityOrder: + For resources with multiple interfaces, the priority order should be selected. + Lower the numerical number, higher the priority + +

Struct: ComputeResourceDescription

+ + + + + + + + + + + + +

Key	Field	Type	Requiredness	Default value
1	computeResourceId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	hostName	`string`	required
3	hostAliases	`list<string>`	optional
4	ipAddresses	`list<string>`	optional
5	resourceDescription	`string`	optional
6	enabled	`bool`	optional
7	batchQueues	`list<BatchQueue>`	optional
8	fileSystems	`map<FileSystems, string>`	optional
9	jobSubmissionInterfaces	`list<JobSubmissionInterface>`	optional
10	dataMovementInterfaces	`list<data_movement_models.DataMovementInterface>`	optional
11	maxMemoryPerNode	`i32`	optional

Computational Resource Description + +computeResourceId: Airavata Internal Unique Identifier to distinguish Compute Resource. + +hostName: + Fully Qualified Host Name. + +hostAliases: + Aliases if any. + +ipAddress: + IP Addresses of the Resource. + +resourceDescription: + A user friendly description of the resource. + +JobSubmissionProtocols: + A computational resources may have one or more ways of submitting Jobs. This structure + will hold all available mechanisms to interact with the resource. + The key is the priority + +DataMovementProtocol: + Option to specify a prefered data movement mechanism of the available options. + +fileSystems: + Map of file systems type and the path. + +

Added: airavata/site/trunk/content/documentation/api-docs/0.16/data_movement_models.html URL: http://svn.apache.org/viewvc/airavata/site/trunk/content/documentation/api-docs/0.16/data_movement_models.html?rev=1720388&view=auto ============================================================================== --- airavata/site/trunk/content/documentation/api-docs/0.16/data_movement_models.html (added) +++ airavata/site/trunk/content/documentation/api-docs/0.16/data_movement_models.html Wed Dec 16 16:25:38 2015 @@ -0,0 +1,144 @@ + + + + + +Thrift module: data_movement_models +

Thrift module: data_movement_models

+ + + + + +

Module	Services	Data types	Constants
data_movement_models		DMType +DataMovementInterface +DataMovementProtocol +GridFTPDataMovement +LOCALDataMovement +SCPDataMovement +SecurityProtocol +UnicoreDataMovement +

Enumerations

Enumeration: DMType

+
+ + +

`COMPUTE_RESOURCE`	`0`	+
`STORAGE_RESOURCE`	`1`	+

Enumeration: SecurityProtocol

+Enumeration of security sshKeyAuthentication and authorization mechanisms supported by Airavata. This enumeration just + describes the supported mechanism. The corresponding security credentials are registered with Airavata Credential + store. + +USERNAME_PASSWORD: + A User Name. + +SSH_KEYS: + SSH Keys + +FIXME: Change GSI to a more precise generic security protocol - X509 + +

+ + + + + + +

`USERNAME_PASSWORD`	`0`	+
`SSH_KEYS`	`1`	+
`GSI`	`2`	+
`KERBEROS`	`3`	+
`OAUTH`	`4`	+
`LOCAL`	`5`	+

Enumeration: DataMovementProtocol

+Enumeration of data movement supported by Airavata + +SCP: + Job manager supporting the Portal Batch System (PBS) protocol. Some examples include TORQUE, PBSPro, Grid Engine. + +SFTP: + The Simple Linux Utility for Resource Management is a open source workload manager. + +GridFTP: + Globus File Transfer Protocol + +UNICORE_STORAGE_SERVICE: + Storage Service Provided by Unicore + +

+ + + + + +

`LOCAL`	`0`	+
`SCP`	`1`	+
`SFTP`	`2`	+
`GridFTP`	`3`	+
`UNICORE_STORAGE_SERVICE`	`4`	+

Data structures

Struct: SCPDataMovement

+ + + + + +

Key	Field	Type	Requiredness	Default value
1	dataMovementInterfaceId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	securityProtocol	`SecurityProtocol`	required
3	alternativeSCPHostName	`string`	optional
4	sshPort	`i32`	optional	`22`

Data Movement through Secured Copy + +alternativeSCPHostName: + If the login to scp is different than the hostname itself, specify it here + +sshPort: + If a non-default port needs to used, specify it. +

Struct: GridFTPDataMovement

+ + + + +

Key	Field	Type	Requiredness	Default value
1	dataMovementInterfaceId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	securityProtocol	`SecurityProtocol`	required
3	gridFTPEndPoints	`list<string>`	required

Data Movement through GridFTP + +alternativeSCPHostName: + If the login to scp is different than the hostname itself, specify it here + +sshPort: + If a non-default port needs to used, specify it. +

Struct: UnicoreDataMovement

+ + + + +

Key	Field	Type	Requiredness	Default value
1	dataMovementInterfaceId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	securityProtocol	`SecurityProtocol`	required
3	unicoreEndPointURL	`string`	required

Data Movement through UnicoreStorage + +unicoreEndPointURL: + unicoreGateway End Point. The provider will query this service to fetch required service end points. +

Struct: LOCALDataMovement

+ + +

Key	Field	Type	Description	Requiredness	Default value
1	dataMovementInterfaceId	`string`		required	`"DO_NOT_SET_AT_CLIENTS"`

LOCAL + +alternativeSCPHostName: + If the login to scp is different than the hostname itself, specify it here + +sshPort: + If a non-defualt port needs to used, specify it. +

Struct: DataMovementInterface

+ + + + +

Key	Field	Type	Requiredness	Default value
1	dataMovementInterfaceId	`string`	required
2	dataMovementProtocol	`DataMovementProtocol`	required
3	priorityOrder	`i32`	required	`0`

Data Movement Interfaces + +dataMovementInterfaceId: The Data Movement Interface has to be previously registered and referenced here. + +priorityOrder: + For resources with multiple interfaces, the priority order should be selected. + Lower the numerical number, higher the priority + +

Added: airavata/site/trunk/content/documentation/api-docs/0.16/experiment_model.html URL: http://svn.apache.org/viewvc/airavata/site/trunk/content/documentation/api-docs/0.16/experiment_model.html?rev=1720388&view=auto ============================================================================== --- airavata/site/trunk/content/documentation/api-docs/0.16/experiment_model.html (added) +++ airavata/site/trunk/content/documentation/api-docs/0.16/experiment_model.html Wed Dec 16 16:25:38 2015 @@ -0,0 +1,122 @@ + + + + + +Thrift module: experiment_model +

Thrift module: experiment_model

+ + + + + +

Module	Services	Data types	Constants
experiment_model		ExperimentModel +ExperimentSearchFields +ExperimentStatistics +ExperimentSummaryModel +ExperimentType +UserConfigurationDataModel +

Enumerations

Enumeration: ExperimentType

+
+ + +

`SINGLE_APPLICATION`	`0`	+
`WORKFLOW`	`1`	+

Enumeration: ExperimentSearchFields

+
+ + + + + + + +

`EXPERIMENT_NAME`	`0`	+
`EXPERIMENT_DESC`	`1`	+
`APPLICATION_ID`	`2`	+
`FROM_DATE`	`3`	+
`TO_DATE`	`4`	+
`STATUS`	`5`	+
`PROJECT_ID`	`6`	+

Data structures

Struct: UserConfigurationDataModel

+ + + + + + + + + +

Key	Field	Type	Requiredness	Default value
1	airavataAutoSchedule	`bool`	required	`false`
2	overrideManualScheduledParams	`bool`	required	`false`
3	shareExperimentPublicly	`bool`	optional	`false`
4	computationalResourceScheduling	`scheduling_model.ComputationalResourceSchedulingModel`	optional
5	throttleResources	`bool`	optional	`false`
6	userDN	`string`	optional
7	generateCert	`bool`	optional	`false`
8	storageId	`string`	optional

A structure holding the experiment configuration. + + +

Struct: ExperimentModel

+ + + + + + + + + + + + + + + + + + + +

Key	Field	Type	Requiredness	Default value
1	experimentId	`string`	required	`"DO_NOT_SET_AT_CLIENTS"`
2	projectId	`string`	required
3	gatewayId	`string`	required
4	experimentType	`ExperimentType`	required	`ExperimentType.SINGLE_APPLICATION`
5	userName	`string`	required
6	experimentName	`string`	required
7	creationTime	`i64`	optional
8	description	`string`	optional
9	executionId	`string`	optional
10	gatewayExecutionId	`string`	optional
11	enableEmailNotification	`bool`	optional
12	emailAddresses	`list<string>`	optional
13	userConfigurationData	`UserConfigurationDataModel`	optional
14	experimentInputs	`list<application_io_models.InputDataObjectType>`	optional
15	experimentOutputs	`list<application_io_models.OutputDataObjectType>`	optional
16	experimentStatus	`status_models.ExperimentStatus`	optional
17	errors	`list<airavata_commons.ErrorModel>`	optional
18	processes	`list<process_model.ProcessModel>`	optional

A structure holding the experiment metadata and its child models. + +userName: + The user name of the targeted gateway end user on whose behalf the experiment is being created. + the associated gateway identity can only be inferred from the security hand-shake so as to avoid + authorized Airavata Clients mimicking an unauthorized request. If a gateway is not registered with + Airavata, an authorization exception is thrown. + +experimentName: + The name of the experiment as defined by the user. The name need not be unique as uniqueness is enforced + by the generated experiment id. + +experimentDescription: + The verbose description of the experiment. This is an optional parameter. +

Struct: ExperimentSummaryModel

+ + + + + + + + + + + + +

Key	Field	Type	Requiredness
1	experimentId	`string`	required
2	projectId	`string`	required
3	gatewayId	`string`	required
4	creationTime	`i64`	optional
5	userName	`string`	required
6	name	`string`	required
7	description	`string`	optional
8	executionId	`string`	optional
9	resourceHostId	`string`	optional
10	experimentStatus	`string`	optional
12	statusUpdateTime	`i64`	optional

Struct: ExperimentStatistics

+ + + + + + + + + + + + + +

Key	Field	Type	Requiredness
1	allExperimentCount	`i32`	required
2	completedExperimentCount	`i32`	required
3	cancelledExperimentCount	`i32`	optional
4	failedExperimentCount	`i32`	required
5	createdExperimentCount	`i32`	required
6	runningExperimentCount	`i32`	required
7	allExperiments	`list<ExperimentSummaryModel>`	required
8	completedExperiments	`list<ExperimentSummaryModel>`	optional
9	failedExperiments	`list<ExperimentSummaryModel>`	optional
10	cancelledExperiments	`list<ExperimentSummaryModel>`	optional
11	createdExperiments	`list<ExperimentSummaryModel>`	optional
12	runningExperiments	`list<ExperimentSummaryModel>`	optional

Added: airavata/site/trunk/content/documentation/api-docs/0.16/gateway_resource_profile_model.html URL: http://svn.apache.org/viewvc/airavata/site/trunk/content/documentation/api-docs/0.16/gateway_resource_profile_model.html?rev=1720388&view=auto ============================================================================== --- airavata/site/trunk/content/documentation/api-docs/0.16/gateway_resource_profile_model.html (added) +++ airavata/site/trunk/content/documentation/api-docs/0.16/gateway_resource_profile_model.html Wed Dec 16 16:25:38 2015 @@ -0,0 +1,86 @@ + + + + + +Thrift module: gateway_resource_profile_model +

Thrift module: gateway_resource_profile_model

+ + + + + +

Module	Services	Data types	Constants
gateway_resource_profile_model		ComputeResourcePreference +GatewayResourceProfile +StoragePreference +

Data structures

Struct: ComputeResourcePreference

+ + + + + + + + + + +

Key	Field	Type	Requiredness	Default value
1	computeResourceId	`string`	required
2	overridebyAiravata	`bool`	required	`true`
3	loginUserName	`string`	optional
4	preferredJobSubmissionProtocol	`compute_resource_model.JobSubmissionProtocol`	optional
5	preferredDataMovementProtocol	`data_movement_models.DataMovementProtocol`	optional
6	preferredBatchQueue	`string`	optional
7	scratchLocation	`string`	optional
8	allocationProjectNumber	`string`	optional
9	resourceSpecificCredentialStoreToken	`string`	optional

Gateway specific preferences for a Computer Resource + +computeResourceId: + Corelate the preference to a compute resource. + +overridebyAiravata: + If turned true, Airavata will override the preferences of better alternatives exist. + +loginUserName: + If turned true, Airavata will override the preferences of better alternatives exist. + +preferredJobSubmissionProtocol: + For resources with multiple job submission protocols, the gateway can pick a preferred option. + +preferredDataMovementProtocol: + For resources with multiple data movement protocols, the gateway can pick a preferred option. + +preferredBatchQueue: + Gateways can choose a defualt batch queue based on average job dimention, reservations or other metrics. + +scratchLocation: + Path to the local scratch space on a HPC cluster. Typically used to create working directory for job execution. + +allocationProjectNumber: + Typically used on HPC machines to charge computing usage to a account number. For instance, on XSEDE once an + allocation is approved, an allocation number is assigned. Before passing this number with job submittions, the + account to be used has to be added to the allocation. + +resourceSpecificCredentialStoreToken: + Resource specific credential store token. If this token is specified, then it is superceeded by the gateway's + default credential store. + +

Struct: StoragePreference

+ + + + + +

Key	Field	Type	Requiredness
1	storageResourceId	`string`	required
2	loginUserName	`string`	optional
3	fileSystemRootLocation	`string`	optional
4	resourceSpecificCredentialStoreToken	`string`	optional

Struct: GatewayResourceProfile

+ + + + + +

Key	Field	Type	Requiredness
1	gatewayID	`string`	required
2	credentialStoreToken	`string`	optional
3	computeResourcePreferences	`list<ComputeResourcePreference>`	optional
4	storagePreferences	`list<StoragePreference>`	optional

Gateway Resource Profile + +gatewayID: + Unique identifier for the gateway assigned by Airavata. Corelate this to Airavata Admin API Gateway Registration. + +credentialStoreToken: + Gateway's defualt credential store token. + +computeResourcePreferences: + List of resource preferences for each of the registered compute resources. + +