airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nipun Udara <y.b.n.ud...@gmail.com>
Subject Re: Clarification about Airavata GFac
Date Wed, 05 Mar 2014 18:30:28 GMT
Hi saminda and Heshan,

Thank you very much for the detailed explanation.


On Wed, Mar 5, 2014 at 11:04 PM, Heshan Suriyaarachchi <
heshan.suriyaarachchi@gmail.com> wrote:

> Hello Nipun,
>
> Please see my inline comments.
>
>
> On Tue, Mar 4, 2014 at 10:32 AM, Nipun Udara <y.b.n.udara@gmail.com>wrote:
>
>> Hi all,
>>
>> I was trying to understand GFac module and found different providers
>> which use to execute jobs like BESProvider and GRAMProvider which extends
>> AbstractProvider class like to know the use of those.
>>
>> And when search abouth EC2 instances there are three different kind of
>> instances such as reserved instances ,spot instances. i would like to know
>> out of those which kind is using in GFac-ec2 module.
>>
>
> As you might know, Amazon provides a java-sdk to communicate with it's
> AWS. EC2Provdier uses it to interact with AMIs and instances. When testing
> out or running a workflow, I usually used a general purpose instance as I
> did not have any requirement for running a workflow in compute optimized
> instance family. Having said that, you will be able to run a workflow in
> any kind of instance that you want. You might have to tweak the UI a bit to
> support this (ie. like adding an option to a drop down, etc and modifying
> the backend a bit).
>
> As for the sample workouts that you might be planning to run, you don't
> need to run them in compute optimized instances as your workflow might not
> be using the full potential of the resources. Keep in mind you are paying
> for the instance type as well. As you reserve an instance with higher
> compute power, you might be paying more (You might want to check the
> pricing of instances before running sample workflows in compute optimized
> instances).
>
> So, the short answer to your question might be, you can use which ever
> instance type you want as long as the AWS java-sdk api is exposing the
> capability of interacting with those instances (which they do).
>
> And is it need to configure to pass user-data scripts that runs the job
>> each time launch a job. I would be grateful if somebody can clarify
>> regarding these.
>>
>
> Let me tell you how the EC2 job submissions work in Airavata.
>
> First you should have a base image which has your application installed.
> This application can be a jar/shell script or other kind of a script. Try
> and wrap your application with a shell script and make it available for
> shell execution.
>
> The EC2 Provider will get the user information (secret key, access key,
> AMI id) and start up an instance from the base image (which has your
> application installed.) Once the instance is running, the EC2 provider
> establishes a SSH connection to the running instance and then runs the
> shell script (which wraps your application) with the user given input
> params. Provider will poll the standard out from the script and extract the
> result out. Then it will be returned to the provider. Then this will passed
> on to the workflow.
>
> AFAICR I used J2ssh for the ssh connections. Since you are getting the
> shell access to the instance, you might be able to run any script type that
> you want but please remember that if you are using any script that is
> dependent on the environment/version, you might run into some issues (eg.
> Running a python 3.0 script on a Python 2.5 system). These things are also
> minor issues and can be easily overcome. The reason why I told you this is
> that, then you would not get side tracked because of any environment
> issues.
>
> I didn't check Airavata mailing list for a long time. Therefore, didn't
> see this. :) Lahiru, pointed me to this thread. If you have any
> questions/doubts, please don't hesitate to ask.
>
> Thanks,
> Heshan.
>
>>
>> Regards
>> Nipun Udara
>>
>> *Undergraduate*
>> *Department of Computer Science & Engineering*
>> *University of Moratuwa*
>> *Sri Lanka*
>>
>
>
>
> --
> Regards,
> Heshan Suriyaarachchi
>
> http://heshans.blogspot.com/
> http://www.linkedin.com/in/heshan
>



-- 
Nipun Udara

*Undergraduate*
*Department of Computer Science & Engineering*
*University of Moratuwa*
*Sri Lanka*

Mime
View raw message