Return-Path: X-Original-To: apmail-airavata-dev-archive@www.apache.org Delivered-To: apmail-airavata-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 044C2111AC for ; Sat, 31 May 2014 15:39:46 +0000 (UTC) Received: (qmail 51630 invoked by uid 500); 31 May 2014 15:39:45 -0000 Delivered-To: apmail-airavata-dev-archive@airavata.apache.org Received: (qmail 51581 invoked by uid 500); 31 May 2014 15:39:45 -0000 Mailing-List: contact dev-help@airavata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airavata.apache.org Delivered-To: mailing list dev@airavata.apache.org Received: (qmail 51574 invoked by uid 99); 31 May 2014 15:39:45 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 31 May 2014 15:39:45 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of gagandeepjuneja@gmail.com designates 209.85.215.49 as permitted sender) Received: from [209.85.215.49] (HELO mail-la0-f49.google.com) (209.85.215.49) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 31 May 2014 15:39:42 +0000 Received: by mail-la0-f49.google.com with SMTP id pv20so1664978lab.36 for ; Sat, 31 May 2014 08:39:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=/sfqVTNsAfM5zqxotvVvDANNkrLngFteY/JwybR9HEQ=; b=JJOZU7w7mtNtcB0tl9SbOX7wh9TyzuUfMxv0RABctE+bx8u7eDkdTTc5tVqwIctbI9 y0nHcv4kPbKYCGD2Zw6DMuWTAPwxL2sklMQ1RtfZDnsWWyEu4lFtcfLr1hMydiA/zrSf +B7ktIejKoeErHQDPtepFvxJLynibzpVP8c322mNvKfj3TRgKW0cgS/sh86lsNgAuwhR 3IX4FuRwSxkd2pdG9uE+yMja0cOF9aPjAORUI7eVqE4fAizpSvIti4/t3epx0Xcy92We pFyubvaiqb2E5+oVd+2mFyF1fq2UGieRUEMUEbhQngcfg3x/J7CUhepzNVE9+EzSl2UG yU7Q== MIME-Version: 1.0 X-Received: by 10.112.180.225 with SMTP id dr1mr18018947lbc.51.1401550760384; Sat, 31 May 2014 08:39:20 -0700 (PDT) Received: by 10.152.130.97 with HTTP; Sat, 31 May 2014 08:39:20 -0700 (PDT) In-Reply-To: References: <53822FE7.1020300@iu.edu> <53836469.101@iu.edu> <5383795B.5050104@iu.edu> Date: Sat, 31 May 2014 21:09:20 +0530 Message-ID: Subject: Re: Need Help !! From: Gagan Juneja To: dev@airavata.apache.org Content-Type: multipart/alternative; boundary=001a11c34940351ce504fab3f775 X-Virus-Checked: Checked by ClamAV on apache.org --001a11c34940351ce504fab3f775 Content-Type: text/plain; charset=UTF-8 Hi Team, Where do we keep the output of a job? Is it somewhere in our database, given back to user or configurable behavior? Regards, Gagan On Tue, May 27, 2014 at 9:49 PM, Gagan Juneja wrote: > Thanks Suresh!! for sharing details about EC2 integration stuff, I am > reading these resources. I would love to contribute in designing and > implementing new gen XBaya but before that I want to have complete > understanding of all the modules in Airavata so that I can contribute in > much effective way. > > > > Regards, > Gagan > > > On Mon, May 26, 2014 at 11:21 PM, Suresh Marru wrote: > >> Hi Gagan, >> >> It was nice talking to you at ApacheCon. We will appreciate your >> contributions. There is a ongoing Google Summer of code project by Nipun >> Udara to revisit EC2 integration. We are moving towards integrating JClouds >> API. Some examples of Amazon use of Airavata can be found at [1] [2] [3]. >> >> If you are looking for something concrete to contribute, one important >> one will be to determine the next generation XBaya. Are you interested? If >> so, please sign up to Airavata Architecture mailing list ( >> http://airavata.apache.org/community/mailing-lists.html) and I will >> start a discussion there. >> >> Suresh >> [1] - http://www.biomedcentral.com/content/pdf/2047-2501-1-6.pdf >> [2] - http://www.cs.ucsb.edu/~rich/publications/book2010.pdf#page=324 >> [3] - >> http://grids.ucs.indiana.edu/ptliupages/publications/biovlab-mmia.v9.pdf >> >> >> On May 26, 2014, at 1:26 PM, Marlon Pierce wrote: >> >> > Hi Gagan-- >> > >> > I think you are looking for GFAC, which provides the connection between >> > Airavata and the backend resource. There is a description of GFAC here >> > [1]. In short, GFAC has two cooperating plugins: a provider is a client >> > to a remote resource (Grid, cloud, etc) and a handler supports the >> > provider to do specialized tasks for a specific scenario. >> > >> > We have a simple tutorial document at [2] on writing handlers (this >> > should move to the wiki soon), and you can find several example >> > providers in the GFAC module of Airavata. >> > >> > Marlon >> > >> > [1] >> > >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=40511561 >> > >> > [2] >> > >> https://docs.google.com/document/d/1RdR2vUIGzV_Nd0wBBCwmzE-Lpc6i6z_jtOgD1L4wUQ4/edit >> > >> > >> > On 5/26/14 1:15 PM, Gagan Juneja wrote: >> >> Hi Marlon, >> >> >> >> This make sense. I will take some time to understand PHP code but >> meanwhile >> >> what I understood is We have two apis. >> >> >> >> Client api helps us in creating Experiment and other stuff and put them >> >> into some database. >> >> Orchestrator api helps in submitting these experiments as jobs for >> >> execution. >> >> >> >> Obviously we are going to perform some business logic or some >> computational >> >> stuff on input data that we set as part of experiment. But where are we >> >> writing that code? >> >> >> >> >> >> As far as Cloud is concern do let me know when you think of running >> this >> >> over Amazon Cloud. I am working with Amazon cloud for last 6 months and >> >> have good experience on automating stuff as well like one click EC2 >> cluster >> >> spawning etc. >> >> >> >> >> >> Regards, >> >> Gagan >> >> >> >> >> >> On Mon, May 26, 2014 at 9:27 PM, Marlon Pierce >> wrote: >> >> >> >>> Hi Gagan-- >> >>> >> >>> Thanks for the questions and please keep them coming. >> >>> >> >>> Airavata 0.11 from late last year is our latest release, and the trunk >> >>> is now very different. Airavata 0.11 is the last version with support >> >>> for XBaya and an older version of the client API (developed mostly by >> >>> Saminda) that has full support for workflows. >> >>> >> >>> We made decisions after Airavata 0.11 to make some major changes. >> >>> >> >>> * Concentrate on Science Gateway use cases at the expense of workflows >> >>> in the near term. Gateways typically run single jobs through a web >> >>> browser. Scaling and multi-tenancy are the challenges. Hopefully >> "near >> >>> term" is drawing to a close. >> >>> >> >>> * Introduce a new component called the Orchestrator that manages job >> >>> submissions and provide a simpler mechanism for single job submissions >> >>> (see previous bullet). Previously, a user would need to define an >> >>> entire workflow for a single job, which was pretty complicated for >> most >> >>> of our driving use cases. >> >>> >> >>> * Define our API in Thrift. This has several advantages in addition to >> >>> the multi-language support. Thrift can support richer data models, >> API >> >>> methods, and exceptions than REST, which was a good fit for our >> project. >> >>> >> >>> * Make significant changes to the Registry to support the new API. >> >>> >> >>> * Put workflow support on lower priority until the Thrift API, >> Registry, >> >>> and Orchestrator were all stable. >> >>> >> >>> These have resulted in several changes that are finally getting >> wrapped >> >>> up and will be released in Airavata 0.12 (tentatively in mid-June). >> >>> After this, we will get back to more frequent releases and also >> >>> resurrect the workflow work. Besides workflows, revising the way >> >>> Airavata manages its application and resource descriptions (called the >> >>> Application Catalog) will be a big focus. >> >>> >> >>> Airavata typically gets used to submit jobs to Grids like the NSF's >> >>> XSEDE, which use Globus GRAM, UNICORE, or GSI-SSH clients. Other >> >>> resources (clouds, the Open Science Grid, non-US Grids) are >> interesting >> >>> to us but we just don't have the resources to look at these much. >> These >> >>> are interesting places for contributions. >> >>> >> >>> At this moment, the most actively developed Thrift clients are >> >>> temporarily outside Airavata's Git repo, although these should be >> merged >> >>> back soon: >> >>> >> >>> * https://github.com/SciGaP/Airavata-PHP-Client-Samples >> >>> >> >>> * https://github.com/SciGaP/PHP-Reference-Gateway/ >> >>> >> >>> Marlon >> >>> >> >>> >> >>> On 5/26/14 11:30 AM, Gagan Juneja wrote: >> >>>> Thanks Saminda and Marlon for your prompt response on my query. >> >>>> >> >>>> I understand from Airavata is "Airavata is a platform which helps >> user to >> >>>> execute his workflow over any distributed environment it could be >> Hadoop >> >>> or >> >>>> anything else of same sort." Is this understanding correct. >> >>>> >> >>>> I am having following question. >> >>>> 1. How this project is being used by the users or client any brief >> idea. >> >>>> Are we exposing cloud, grids as a service or user needs to implement >> on >> >>>> their on. >> >>>> 2. I have looked at class CreateLaunchExperimentUS3.java. Here most >> of >> >>> the >> >>>> work is like creating an experiment and launching it. I am not able >> to >> >>>> find any thing related to workflow or job that we want to run this >> input >> >>>> data that we are setting in Experiment object. Where can we create >> Jobs, >> >>>> tasks or workflows? >> >>>> 3. Why do Xbaya is deprecated and replaced with thrift API? (Anything >> >>>> special apart from multi language support.) >> >>>> >> >>>> >> >>>> I have lot more question will keep bothering you :). >> >>>> >> >>>> Regards, >> >>>> Gagan >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> On Sun, May 25, 2014 at 11:31 PM, Marlon Pierce >> wrote: >> >>>> >> >>>>> The QuickStart info is out of date, and we will be updating it soon. >> >>>>> After compiling and starting the server, you can try out PHP sample >> >>>>> scripts in >> >>>>> >> >>>>> >> >>> >> ./airavata-api/airavata-client-sdks/airavata-php-sdk/src/main/resources/samples/ >> >>>>> >> >>>>> Marlon >> >>>>> >> >>>>> On 5/24/14 4:13 AM, Saminda Wijeratne wrote: >> >>>>>> XBaya is deprecated in the upcoming release. Please find all the >> >>> release >> >>>>>> artifacts under >> modules/distribution/release/target/release-artifacts. >> >>>>>> >> >>>>>> >> >>>>>> On Fri, May 23, 2014 at 11:14 PM, Gagan Juneja < >> >>>>> gagandeepjuneja@gmail.com>wrote: >> >>>>>>> Hi Team, >> >>>>>>> I am new to Airavata project. I was following Quick Start >> document. I >> >>>>>>> build Airavata project using default profile. I did not find any >> >>>>> artifact >> >>>>>>> in modules/distribution target directory and even many of the >> modules >> >>>>> did >> >>>>>>> not build such as xbaya. >> >>>>>>> >> >>>>>>> Please guide me how to start with this. >> >>>>>>> >> >>>>>>> Thanks & Regards, >> >>>>>>> Gagan >> >>>>>>> >> >>> >> > >> >> > --001a11c34940351ce504fab3f775 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi Team,

Where do we keep the output of= a job? Is it somewhere in our database, given back to user or configurable= behavior?


Regards,
Gagan=


On Tue,= May 27, 2014 at 9:49 PM, Gagan Juneja <gagandeepjuneja@gmail.com<= /a>> wrote:
Thanks Suresh!! for sharing= details about EC2 integration stuff, I am reading these resources. I would= love to contribute in designing and implementing new gen XBaya but before = that I want to have complete understanding of all the modules in Airavata s= o that I can contribute in much effective way.



Regards,
Gagan
=

On Mon, May 26, 2014 at 11:21 PM, Suresh M= arru <smarru@apache.org> wrote:
Hi Gagan,

It was nice talking to you at ApacheCon. We will appreciate your contributi= ons. There is a ongoing Google Summer of code project by Nipun Udara to rev= isit EC2 integration. We are moving towards integrating JClouds API. Some e= xamples of Amazon use of Airavata can be found at [1] [2] [3].

If you are looking for something concrete to contribute, one important one = will be to determine the next generation XBaya. Are you interested? If so, = please sign up to Airavata Architecture mailing list (http://air= avata.apache.org/community/mailing-lists.html) and I will start a discu= ssion there.

Suresh
[1] - http://www.biomedcentral.com/content/pdf/2047-2501-1-6.= pdf
[2] - http://www.cs.ucsb.edu/~rich/publications/book20= 10.pdf#page=3D324
[3] - http://grids.ucs.indiana.edu/ptliupages/p= ublications/biovlab-mmia.v9.pdf


On May 26, 2014, at 1:26 PM, Marlon Pierce <marpierc@iu.edu> wrote:

> Hi Gagan--
>
> I think you are looking for GFAC, which provides the connection betwee= n
> Airavata and the backend resource. =C2=A0There is a description of GFA= C here
> [1]. =C2=A0In short, GFAC has two cooperating plugins: a provider is a= client
> to a remote resource (Grid, cloud, etc) and a handler supports the
> provider to do specialized tasks for a specific scenario.
>
> We have a simple tutorial document at [2] on writing handlers (this > should move to the wiki soon), and you can find several example
> providers in the GFAC module of Airavata.
>
> Marlon
>
> [1]
> https://cwiki.apache.org/confluence/pag= es/viewpage.action?pageId=3D40511561
>
> [2]
> https://docs.google.com/docume= nt/d/1RdR2vUIGzV_Nd0wBBCwmzE-Lpc6i6z_jtOgD1L4wUQ4/edit
>
>
> On 5/26/14 1:15 PM, Gagan Juneja wrote:
>> Hi Marlon,
>>
>> This make sense. I will take some time to understand PHP code but = meanwhile
>> what I understood is We have two apis.
>>
>> Client api helps us in creating Experiment and other stuff and put= them
>> into some database.
>> Orchestrator api helps in submitting these experiments as jobs for=
>> execution.
>>
>> Obviously we are going to perform some business logic or some comp= utational
>> stuff on input data that we set as part of experiment. But where a= re we
>> writing that code?
>>
>>
>> As far as Cloud is concern do let me know when you think of runnin= g this
>> over Amazon Cloud. I am working with Amazon cloud for last 6 month= s and
>> have good experience on automating stuff as well like one click EC= 2 cluster
>> spawning etc.
>>
>>
>> Regards,
>> Gagan
>>
>>
>> On Mon, May 26, 2014 at 9:27 PM, Marlon Pierce <marpierc@iu.edu> wrote:
>>
>>> Hi Gagan--
>>>
>>> Thanks for the questions and please keep them coming.
>>>
>>> Airavata 0.11 from late last year is our latest release, and t= he trunk
>>> is now very different. Airavata 0.11 is the last version with = support
>>> for XBaya and an older version of the client API (developed mo= stly by
>>> Saminda) that has full support for workflows.
>>>
>>> We made decisions after Airavata 0.11 to make some major chang= es.
>>>
>>> * Concentrate on Science Gateway use cases at the expense of w= orkflows
>>> in the near term. Gateways typically run single jobs through a= web
>>> browser. Scaling and multi-tenancy are the challenges. =C2=A0H= opefully "near
>>> term" is drawing to a close.
>>>
>>> * Introduce a new component called the Orchestrator that manag= es job
>>> submissions and provide a simpler mechanism for single job sub= missions
>>> (see previous bullet). =C2=A0Previously, a user would need to = define an
>>> entire workflow for a single job, which was pretty complicated= for most
>>> of our driving use cases.
>>>
>>> * Define our API in Thrift. This has several advantages in add= ition to
>>> the multi-language support. =C2=A0Thrift can support richer da= ta models, API
>>> methods, and exceptions than REST, which was a good fit for ou= r project.
>>>
>>> * Make significant changes to the Registry to support the new = API.
>>>
>>> * Put workflow support on lower priority until the Thrift API,= Registry,
>>> and Orchestrator were all stable.
>>>
>>> These have resulted in several changes that are finally gettin= g wrapped
>>> up and will be released in Airavata 0.12 (tentatively in mid-J= une).
>>> After this, we will get back to more frequent releases and als= o
>>> resurrect the workflow work. =C2=A0Besides workflows, revising= the way
>>> Airavata manages its application and resource descriptions (ca= lled the
>>> Application Catalog) will be a big focus.
>>>
>>> Airavata typically gets used to submit jobs to Grids like the = NSF's
>>> XSEDE, which use Globus GRAM, UNICORE, or GSI-SSH clients. =C2= =A0Other
>>> resources (clouds, the Open Science Grid, non-US Grids) are in= teresting
>>> to us but we just don't have the resources to look at thes= e much. These
>>> are interesting places for contributions.
>>>
>>> At this moment, the most actively developed Thrift clients are=
>>> temporarily outside Airavata's Git repo, although these sh= ould be merged
>>> back soon:
>>>
>>> * https://github.com/SciGaP/Airavata-PHP-Client-Sampl= es
>>>
>>> * https://github.com/SciGaP/PHP-Reference-Gateway/
>>>
>>> Marlon
>>>
>>>
>>> On 5/26/14 11:30 AM, Gagan Juneja wrote:
>>>> Thanks Saminda and Marlon for your prompt response on my q= uery.
>>>>
>>>> I understand from Airavata is "Airavata is a platform= which helps user to
>>>> execute his workflow over any distributed environment it c= ould be Hadoop
>>> or
>>>> anything else of same sort." Is this understanding co= rrect.
>>>>
>>>> I am having following question.
>>>> 1. How this project is being used by the users or client a= ny brief idea.
>>>> Are we exposing cloud, grids as a service or user needs to= implement on
>>>> their on.
>>>> 2. I have looked at class CreateLaunchExperimentUS3.java. = Here most of
>>> the
>>>> work is like creating an experiment and launching it. =C2= =A0I am not able to
>>>> find any thing related to workflow or job that we want to = run this input
>>>> data that we are setting in Experiment object. Where can w= e create Jobs,
>>>> tasks or workflows?
>>>> 3. Why do Xbaya is deprecated and replaced with thrift API= ? (Anything
>>>> special apart from multi language support.)
>>>>
>>>>
>>>> I have lot more question will keep bothering you :).
>>>>
>>>> Regards,
>>>> Gagan
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Sun, May 25, 2014 at 11:31 PM, Marlon Pierce <marpierc@iu.edu> wrot= e:
>>>>
>>>>> The QuickStart info is out of date, and we will be upd= ating it soon.
>>>>> After compiling and starting the server, you can try o= ut PHP sample
>>>>> scripts in
>>>>>
>>>>>
>>> ./airavata-api/airavata-client-sdks/airavata-php-sdk/src/main/= resources/samples/
>>>>>
>>>>> Marlon
>>>>>
>>>>> On 5/24/14 4:13 AM, Saminda Wijeratne wrote:
>>>>>> XBaya is deprecated in the upcoming release. Pleas= e find all the
>>> release
>>>>>> artifacts under modules/distribution/release/targe= t/release-artifacts.
>>>>>>
>>>>>>
>>>>>> On Fri, May 23, 2014 at 11:14 PM, Gagan Juneja <= ;
>>>>> gagandeepjuneja@gmail.com>wrote:
>>>>>>> Hi Team,
>>>>>>> I am new to Airavata project. I was following = Quick Start document. I
>>>>>>> build Airavata project using default profile. = I did not find any
>>>>> artifact
>>>>>>> in modules/distribution target directory and e= ven many of the modules
>>>>> did
>>>>>>> not build such as xbaya.
>>>>>>>
>>>>>>> Please guide me how to start with this.
>>>>>>>
>>>>>>> Thanks & Regards,
>>>>>>> Gagan
>>>>>>>
>>>
>



--001a11c34940351ce504fab3f775--