airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lahiru Gunathilake <glah...@gmail.com>
Subject Re: ec2 input types
Date Thu, 12 Jun 2014 14:28:11 GMT
Hi Nipun,

I think you can implement this as an handler and if the input/output types
are URI you can invoke these handler logics. If we want to add new schemas
and thrift models we can focus on them little later.

First you can write an handlers to transfer inputs and outputs and write
test cases to test them. Please refer SSHInputHandler,SSHOutputHandler and
AdvancedSCPInputHandler and AdvancedSCPOutputHandler. Initial two handlers
transfer files to the remote resource from airavata host and advance
handlers transfer files from third party machine to remote machine.

Please try to find whether the API allow to do third party transfer, in
that case its easy to have one set of handlers to transfer from remote
machine to S3 (basically two handlers one for input one for output).

Regards
Lahiru


On Thu, Jun 12, 2014 at 12:12 AM, Raminderjeet Singh <ramifnu@indiana.edu>
wrote:

> Hi Nipun,
>
> S3 [1] is the storage service provided by Amazon and it provide persistent
> storage for input/output data. There are Amazon API's provided to develop
> clients[2]. To add S3 input/output type in Airavata, you need to change
> gfac schema and also thrift model. These 2 places will be merged to one in
> 0.13 release. They you can expect client to set input as S3 type and you
> need to add handlers to upload and download the data from S3.
>
> Thanks
> Raminder
>
> 1. http://aws.amazon.com/s3/
> 2.
> http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/index.html?com/amazonaws/services/s3/transfer/TransferManager.html
>
>
> On Wed, Jun 11, 2014 at 11:53 PM, Nipun Udara <y.b.n.udara@gmail.com>
> wrote:
>
>> Hi raminder singh
>>
>> I cannot find S3type among other input types. How can i add S3type can
>> you give a brief idea about doing that.
>>
>> Regards
>>
>>
>> On Wed, Jun 11, 2014 at 8:59 PM, Raminder Singh <raminderjsingh@gmail.com
>> > wrote:
>>
>>> URI and S3 types can be used for data movement. We should support SCP,
>>> HTTP,FTP data transfers under URI type.
>>>
>>> Thanks
>>> Raminder
>>>
>>> On Jun 11, 2014, at 11:10 AM, Nipun Udara <y.b.n.udara@gmail.com> wrote:
>>>
>>> > Hi all
>>> >
>>> > current ec2 implementation use only String type inputs. what other
>>> input types will be used with ec2. i would be grateful if someone can give
>>> a idea.
>>> >
>>> > regards
>>> > Nipun Udara
>>> >
>>> >
>>>
>>>
>>
>>
>> --
>> Nipun Udara
>>
>> *Undergraduate*
>> *Department of Computer Science & Engineering*
>> *University of Moratuwa*
>> *Sri Lanka*
>>
>>
>


-- 
System Analyst Programmer
PTI Lab
Indiana University

Mime
View raw message