airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raminder Singh <raminderjsi...@gmail.com>
Subject Re: Need Help !!
Date Sun, 01 Jun 2014 16:45:38 GMT
Outputs are files, those need to moved to gateway (portal) or user machine. We currently use
GridFTP[1]/SCP/GSISCP libraries for file movement . We explored https://www.globus.org/ service
also some time back but don’t have a direct integration yet. Globus service can allow Airavata
users to move files to and from their workstation. 

Thanks
Raminder

1. http://toolkit.globus.org/toolkit/docs/latest-stable/gridftp/

On May 31, 2014, at 11:39 AM, Gagan Juneja <gagandeepjuneja@gmail.com> wrote:

> Hi Team,
> 
> Where do we keep the output of a job? Is it somewhere in our database, given back to
user or configurable behavior?
> 
> 
> Regards,
> Gagan
> 
> 
> On Tue, May 27, 2014 at 9:49 PM, Gagan Juneja <gagandeepjuneja@gmail.com> wrote:
> Thanks Suresh!! for sharing details about EC2 integration stuff, I am reading these resources.
I would love to contribute in designing and implementing new gen XBaya but before that I want
to have complete understanding of all the modules in Airavata so that I can contribute in
much effective way.
> 
> 
> 
> Regards,
> Gagan
> 
> 
> On Mon, May 26, 2014 at 11:21 PM, Suresh Marru <smarru@apache.org> wrote:
> Hi Gagan,
> 
> It was nice talking to you at ApacheCon. We will appreciate your contributions. There
is a ongoing Google Summer of code project by Nipun Udara to revisit EC2 integration. We are
moving towards integrating JClouds API. Some examples of Amazon use of Airavata can be found
at [1] [2] [3].
> 
> If you are looking for something concrete to contribute, one important one will be to
determine the next generation XBaya. Are you interested? If so, please sign up to Airavata
Architecture mailing list (http://airavata.apache.org/community/mailing-lists.html) and I
will start a discussion there.
> 
> Suresh
> [1] - http://www.biomedcentral.com/content/pdf/2047-2501-1-6.pdf
> [2] - http://www.cs.ucsb.edu/~rich/publications/book2010.pdf#page=324
> [3] - http://grids.ucs.indiana.edu/ptliupages/publications/biovlab-mmia.v9.pdf
> 
> 
> On May 26, 2014, at 1:26 PM, Marlon Pierce <marpierc@iu.edu> wrote:
> 
> > Hi Gagan--
> >
> > I think you are looking for GFAC, which provides the connection between
> > Airavata and the backend resource.  There is a description of GFAC here
> > [1].  In short, GFAC has two cooperating plugins: a provider is a client
> > to a remote resource (Grid, cloud, etc) and a handler supports the
> > provider to do specialized tasks for a specific scenario.
> >
> > We have a simple tutorial document at [2] on writing handlers (this
> > should move to the wiki soon), and you can find several example
> > providers in the GFAC module of Airavata.
> >
> > Marlon
> >
> > [1]
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=40511561
> >
> > [2]
> > https://docs.google.com/document/d/1RdR2vUIGzV_Nd0wBBCwmzE-Lpc6i6z_jtOgD1L4wUQ4/edit
> >
> >
> > On 5/26/14 1:15 PM, Gagan Juneja wrote:
> >> Hi Marlon,
> >>
> >> This make sense. I will take some time to understand PHP code but meanwhile
> >> what I understood is We have two apis.
> >>
> >> Client api helps us in creating Experiment and other stuff and put them
> >> into some database.
> >> Orchestrator api helps in submitting these experiments as jobs for
> >> execution.
> >>
> >> Obviously we are going to perform some business logic or some computational
> >> stuff on input data that we set as part of experiment. But where are we
> >> writing that code?
> >>
> >>
> >> As far as Cloud is concern do let me know when you think of running this
> >> over Amazon Cloud. I am working with Amazon cloud for last 6 months and
> >> have good experience on automating stuff as well like one click EC2 cluster
> >> spawning etc.
> >>
> >>
> >> Regards,
> >> Gagan
> >>
> >>
> >> On Mon, May 26, 2014 at 9:27 PM, Marlon Pierce <marpierc@iu.edu> wrote:
> >>
> >>> Hi Gagan--
> >>>
> >>> Thanks for the questions and please keep them coming.
> >>>
> >>> Airavata 0.11 from late last year is our latest release, and the trunk
> >>> is now very different. Airavata 0.11 is the last version with support
> >>> for XBaya and an older version of the client API (developed mostly by
> >>> Saminda) that has full support for workflows.
> >>>
> >>> We made decisions after Airavata 0.11 to make some major changes.
> >>>
> >>> * Concentrate on Science Gateway use cases at the expense of workflows
> >>> in the near term. Gateways typically run single jobs through a web
> >>> browser. Scaling and multi-tenancy are the challenges.  Hopefully "near
> >>> term" is drawing to a close.
> >>>
> >>> * Introduce a new component called the Orchestrator that manages job
> >>> submissions and provide a simpler mechanism for single job submissions
> >>> (see previous bullet).  Previously, a user would need to define an
> >>> entire workflow for a single job, which was pretty complicated for most
> >>> of our driving use cases.
> >>>
> >>> * Define our API in Thrift. This has several advantages in addition to
> >>> the multi-language support.  Thrift can support richer data models, API
> >>> methods, and exceptions than REST, which was a good fit for our project.
> >>>
> >>> * Make significant changes to the Registry to support the new API.
> >>>
> >>> * Put workflow support on lower priority until the Thrift API, Registry,
> >>> and Orchestrator were all stable.
> >>>
> >>> These have resulted in several changes that are finally getting wrapped
> >>> up and will be released in Airavata 0.12 (tentatively in mid-June).
> >>> After this, we will get back to more frequent releases and also
> >>> resurrect the workflow work.  Besides workflows, revising the way
> >>> Airavata manages its application and resource descriptions (called the
> >>> Application Catalog) will be a big focus.
> >>>
> >>> Airavata typically gets used to submit jobs to Grids like the NSF's
> >>> XSEDE, which use Globus GRAM, UNICORE, or GSI-SSH clients.  Other
> >>> resources (clouds, the Open Science Grid, non-US Grids) are interesting
> >>> to us but we just don't have the resources to look at these much. These
> >>> are interesting places for contributions.
> >>>
> >>> At this moment, the most actively developed Thrift clients are
> >>> temporarily outside Airavata's Git repo, although these should be merged
> >>> back soon:
> >>>
> >>> * https://github.com/SciGaP/Airavata-PHP-Client-Samples
> >>>
> >>> * https://github.com/SciGaP/PHP-Reference-Gateway/
> >>>
> >>> Marlon
> >>>
> >>>
> >>> On 5/26/14 11:30 AM, Gagan Juneja wrote:
> >>>> Thanks Saminda and Marlon for your prompt response on my query.
> >>>>
> >>>> I understand from Airavata is "Airavata is a platform which helps user
to
> >>>> execute his workflow over any distributed environment it could be Hadoop
> >>> or
> >>>> anything else of same sort." Is this understanding correct.
> >>>>
> >>>> I am having following question.
> >>>> 1. How this project is being used by the users or client any brief idea.
> >>>> Are we exposing cloud, grids as a service or user needs to implement
on
> >>>> their on.
> >>>> 2. I have looked at class CreateLaunchExperimentUS3.java. Here most
of
> >>> the
> >>>> work is like creating an experiment and launching it.  I am not able
to
> >>>> find any thing related to workflow or job that we want to run this input
> >>>> data that we are setting in Experiment object. Where can we create Jobs,
> >>>> tasks or workflows?
> >>>> 3. Why do Xbaya is deprecated and replaced with thrift API? (Anything
> >>>> special apart from multi language support.)
> >>>>
> >>>>
> >>>> I have lot more question will keep bothering you :).
> >>>>
> >>>> Regards,
> >>>> Gagan
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Sun, May 25, 2014 at 11:31 PM, Marlon Pierce <marpierc@iu.edu>
wrote:
> >>>>
> >>>>> The QuickStart info is out of date, and we will be updating it soon.
> >>>>> After compiling and starting the server, you can try out PHP sample
> >>>>> scripts in
> >>>>>
> >>>>>
> >>> ./airavata-api/airavata-client-sdks/airavata-php-sdk/src/main/resources/samples/
> >>>>>
> >>>>> Marlon
> >>>>>
> >>>>> On 5/24/14 4:13 AM, Saminda Wijeratne wrote:
> >>>>>> XBaya is deprecated in the upcoming release. Please find all
the
> >>> release
> >>>>>> artifacts under modules/distribution/release/target/release-artifacts.
> >>>>>>
> >>>>>>
> >>>>>> On Fri, May 23, 2014 at 11:14 PM, Gagan Juneja <
> >>>>> gagandeepjuneja@gmail.com>wrote:
> >>>>>>> Hi Team,
> >>>>>>> I am new to Airavata project. I was following Quick Start
document. I
> >>>>>>> build Airavata project using default profile. I did not
find any
> >>>>> artifact
> >>>>>>> in modules/distribution target directory and even many of
the modules
> >>>>> did
> >>>>>>> not build such as xbaya.
> >>>>>>>
> >>>>>>> Please guide me how to start with this.
> >>>>>>>
> >>>>>>> Thanks & Regards,
> >>>>>>> Gagan
> >>>>>>>
> >>>
> >
> 
> 
> 


Mime
View raw message