Return-Path: X-Original-To: apmail-airavata-dev-archive@www.apache.org Delivered-To: apmail-airavata-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BCDE5186EE for ; Mon, 7 Dec 2015 18:09:24 +0000 (UTC) Received: (qmail 45513 invoked by uid 500); 7 Dec 2015 18:09:02 -0000 Delivered-To: apmail-airavata-dev-archive@airavata.apache.org Received: (qmail 45459 invoked by uid 500); 7 Dec 2015 18:09:02 -0000 Mailing-List: contact dev-help@airavata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airavata.apache.org Delivered-To: mailing list dev@airavata.apache.org Received: (qmail 45449 invoked by uid 99); 7 Dec 2015 18:09:02 -0000 Received: from Unknown (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Dec 2015 18:09:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id E6894C1335 for ; Mon, 7 Dec 2015 18:09:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.001 X-Spam-Level: *** X-Spam-Status: No, score=3.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=3, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=binghamton-edu.20150623.gappssmtp.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id T-o0xPXB_XVZ for ; Mon, 7 Dec 2015 18:08:48 +0000 (UTC) Received: from mail-yk0-f180.google.com (mail-yk0-f180.google.com [209.85.160.180]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 5DF1542AA8 for ; Mon, 7 Dec 2015 18:08:48 +0000 (UTC) Received: by ykfs79 with SMTP id s79so205197124ykf.1 for ; Mon, 07 Dec 2015 10:03:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=binghamton-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=1q7Edwuj5xZG/dGaGWp0zCP+GFpn1+yI9mkUPr5p4f8=; b=ZXVbO5twgDI6xnhoZwiRoe2zJeW7O9Kipk5bCmkiq5h7ZkulLMG3qxNfyyr1DmYm1q ySAjQS0+VCSitLQchp5wehNei8Y8Us9Qnij5jDOv6CT4xYdCKTZqs4yyN1L+XCfPmS8Z 5YM7tysuyLkHhwVu0evrA94Kfy/BmFWgtktTFPqsZ4OkVlLYmrFbP1e/TLDUNH8Iyj2J FgrxhtSNlesfnqyExqzP2oEbUThHxr+ivkaFlbVobwtUUki9p0x9/lyULAjRQKB528Xz Z1m5jM4BPLeQCmSSlP/T80lwCpUTi2RYSrGPiOq9OAMQI4yUk2cgWx2/woetzIMlAO9b NuuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=1q7Edwuj5xZG/dGaGWp0zCP+GFpn1+yI9mkUPr5p4f8=; b=avjxB6yDUoGoWsICoi0StL9SPRrOw3ahCuBRaG7oQ1jPDGZXhsoNia1LdhYQUJI+Re KdNpb3jbpnfUj6n3Tgc2bikXO55A9Wrq3f2dES+OkxIc771XqK2bQabDCBR0zJbFKOSv JMv2/KUjDf8uDfCYbUjENVQJNvBLxqBS7Qu6fum47ie2GaFmfqPT2dgkvEbn0MfNEvPR 1IwEPcgniSEE6Gs6UA8bS57ASody3+8RAzC8cwUP+Wh9etU0plNIyBEXXSbV0RD67EB7 0q6XGa+y2NuZz20XvHEyMCLlBxbNjp2YJJITUIe3aWw5mJwKTzlBA/+BghdrT/XKbqpq pyAQ== X-Gm-Message-State: ALoCoQmka78HH644syjQ06qFRCg+X0J/nZJp56XTKZ9OD2G78SEeHnhworLGDThVq56XOvfjlvvw MIME-Version: 1.0 X-Received: by 10.13.219.2 with SMTP id d2mr23286414ywe.344.1449511420497; Mon, 07 Dec 2015 10:03:40 -0800 (PST) Received: by 10.37.80.19 with HTTP; Mon, 7 Dec 2015 10:03:40 -0800 (PST) Received: by 10.37.80.19 with HTTP; Mon, 7 Dec 2015 10:03:40 -0800 (PST) In-Reply-To: <70E70A68-A183-4913-9D10-890D2EC04104@apache.org> References: <309191B8-227F-4FFF-AEDF-67EF3F97461A@illinois.edu> <115C2373-7149-422D-A14D-E23D439A7AD7@apache.org> <70E70A68-A183-4913-9D10-890D2EC04104@apache.org> Date: Mon, 7 Dec 2015 13:03:40 -0500 Message-ID: Subject: Re: Airavata installation to submit a job in Mesos/Maraton cluster From: Pankaj Saha To: dev Content-Type: multipart/alternative; boundary=94eb2c06c59e514659052652ad75 --94eb2c06c59e514659052652ad75 Content-Type: text/plain; charset=UTF-8 Hi Suresh, I am setting up for 0.16 and will implement the approach. Will update you. Thanks Pankaj On 07-Dec-2015 12:50 PM, "Suresh Marru" wrote: > Hi Pankaj, > > Yes this should work as you are trying to do. Lets try and get this > scenario working with 0.16. Can you start updating your PGA and Airavata to > latest git master, that way its easy to debug with active code base. 0.15 > branch is frozen and if we find any bug, it will be easier to fix in > master. > > Suresh > > On Dec 5, 2015, at 6:43 PM, Pankaj Saha wrote: > > HI Suresh, > Yes Now airavata can not submit mesos jobs directl. Here is the plan I was > trying to implement. > my local system has Marathon client installed which is connected to Mesos > a cluster. If I can configure my system as an available computer resource > in the Airavata PGA then I can start an experiment on this. Now that > experiment will run on my local computer which eventually launch a marathon > job through a command with help of marathon rest API. > > Do you think that will work? > > Thanks > Pankaj > > > > On Sat, Dec 5, 2015 at 3:25 PM, Suresh Marru wrote: > >> Hi Pankaj, >> >> You probably already know this, there is no Mesos submission in Airavata >> out of the box or in the version deployed on GW56?. Did you add that >> implementation in your local version? >> >> Suresh >> >> On Dec 5, 2015, at 2:53 PM, Pankaj Saha wrote: >> >> is your mesos cluster registered in the PGA as computer resource. >> - No >> Is the hello world application catalog entries made. >> -No >> Can you do this in a PGA using the gw56 ( I assume you have admin >> privileges). >> - I dont see any option for creating that. But I remember during GSoC, >> Erom explained those PGA pages through which admin can do this. >> >> On Sat, Dec 5, 2015 at 2:49 PM, Pamidighantam, Sudhakar V < >> spamidig@illinois.edu> wrote: >> >>> Pankaj; >>> is your mesos cluster registered in the PGA as computer resource. Is the >>> hello world application catalog entries made. Can you do this in a PGA >>> using the gw56 ( I assume you have admin privileges). If not somebody can >>> do this for you. >>> >>> Thanks, >>> Sudhakar. >>> >>> On Dec 5, 2015, at 1:26 PM, Pankaj Saha wrote: >>> >>> Hi Supun, >>> I have pointed to 'airavata-server' => 'gw56.iu.xsede.org >>> ', >>> to start working with.As you have already provided all the required >>> previleges to my new userid, I am able to see all computer resources and >>> applications in it. So it was not a problem with PGA but Airavata server >>> installation in my local machine. >>> >>> Now can anyone please guide me for launching a hello world application >>> through this portal so that it can launch a job in a mesos cluster. I know >>> that I can not use the "gw56.iu.xsede.org >>> " >>> server, as I want to launch a job in my local mesos/marathon cluster. But >>> if I get some clue how to start with I can possible fix my local Airavata >>> server installation and will try to launch a job in local resource. >>> >>> Thanks >>> Pankaj >>> >>> On Fri, Dec 4, 2015 at 5:41 PM, Supun Nakandala < >>> supun.nakandala@gmail.com> wrote: >>> >>>> We haven't done any changes to 0.15 branches recently. So if it worked >>>> for you previously it should still work without any issue. I suggest that >>>> you first try to setup your pga to work with SciGaP hosted airavata and >>>> then try to connect it to your own airavata. But we don't have any active >>>> deployments of 0.15. In that case you may want to use master branch >>>> >>>> On Fri, Dec 4, 2015 at 5:36 PM, Pankaj Saha >>>> wrote: >>>> >>>>> While running server, no error messages. But while creating projects >>>>> its throws the error message which I have posted. >>>>> On 04-Dec-2015 4:55 PM, "Supun Nakandala" >>>>> wrote: >>>>> >>>>>> do you get any airavata error messages? >>>>>> >>>>>> On Fri, Dec 4, 2015 at 4:51 PM, Pankaj Saha >>>>>> wrote: >>>>>> >>>>>>> I am using 15 branch >>>>>>> yes I could not even create a project. >>>>>>> >>>>>>> On Fri, Dec 4, 2015 at 4:49 PM, Supun Nakandala < >>>>>>> supun.nakandala@gmail.com> wrote: >>>>>>> >>>>>>>> Which Airavata/PGA version are you using? >>>>>>>> >>>>>>>> Looking at your error message I see even the project is also not >>>>>>>> set. >>>>>>>> >>>>>>>> On Fri, Dec 4, 2015 at 12:16 PM, Pankaj Saha >>>>>>> > wrote: >>>>>>>> >>>>>>>>> Hello Devs, >>>>>>>>> I have a requirement to launch a job (simple hello world kind of) >>>>>>>>> on mesos/marathon cluster through Airavata client (PGA preferable ). I have >>>>>>>>> tried installing a local Aiaravata PGA with hosted Airavata server on my >>>>>>>>> machine. Now the problem is with PGA installation, its not allowing me >>>>>>>>> create or explore any project/experiment. >>>>>>>>> Its saying: >>>>>>>>> *Required field 'gatewayId' was not present! Struct: >>>>>>>>> createProject_args(gatewayId:null, project:null)* >>>>>>>>> >>>>>>>>> I always used the default gateway-Id which is "default" and the >>>>>>>>> same is mentioned in the server configuration properties file too. This >>>>>>>>> problem I have never faced earlier and Airavata installation used to be >>>>>>>>> very simple. >>>>>>>>> >>>>>>>>> Second challenge is to run a job through Airavata client, so that >>>>>>>>> it can launch a job in a Mesos cluster through Marathon. So Airavata should >>>>>>>>> be able to invoke a rest API based client of Marathon to do the job. >>>>>>>>> I seek help to resolve the first issue and then need to find a way >>>>>>>>> for launching jobs. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Pankaj >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Thank you >>>>>>>> Supun Nakandala >>>>>>>> Dept. Computer Science and Engineering >>>>>>>> University of Moratuwa >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Thank you >>>>>> Supun Nakandala >>>>>> Dept. Computer Science and Engineering >>>>>> University of Moratuwa >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Thank you >>>> Supun Nakandala >>>> Dept. Computer Science and Engineering >>>> University of Moratuwa >>>> >>> >>> >>> >> >> > > --94eb2c06c59e514659052652ad75 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Hi Suresh,
I am setting up for 0.16=C2=A0 and will implement the approach. Will update= you.

Thanks
Pankaj

On 07-Dec-2015 12:50 PM, "Suresh Marru"= ; <smarru@apache.org> wrote:=
Hi Pankaj,

Yes this should work as you = are trying to do. Lets try and get this scenario working with 0.16. Can you= start updating your PGA and Airavata to latest git master, that way its ea= sy to debug with active code base. 0.15 branch is frozen and if we find any= bug, it will be easier to fix in master.=C2=A0

Su= resh

On Dec 5, 2015, at 6= :43 PM, Pankaj Saha <psaha4@binghamton.edu> wrote:

HI Suresh,
Yes Now airavata can not submit mesos jobs directl. Here is= the plan I was trying to implement.
my local system has Marathon= client installed which is connected to Mesos a cluster. If I can configure= my system as an available computer resource in the Airavata PGA then I can= start an experiment on this. Now that experiment will run on my local comp= uter which eventually launch a marathon job through a command with help of = marathon rest API.

Do you think that will work?

Thanks
Pankaj


On S= at, Dec 5, 2015 at 3:25 PM, Suresh Marru <smarru@apache.org>= wrote:
Hi Pankaj,

You probably already know this, there is n= o Mesos submission in Airavata out of the box or in the version deployed on= GW56?. Did you add that implementation in your local version?=C2=A0
<= span>

Suresh
=

On Dec 5, 2015, at 2= :53 PM, Pankaj Saha <psaha4@binghamton.edu> wrote:

is your mesos cluster re= gistered in the PGA as computer resource.
- No
Is= the hello world application catalog entries made.=C2=A0
<= span style=3D"font-size:12.8000001907349px">-No
Can you do this in a PGA using the gw56 = ( I assume you have admin privileges).=C2=A0
- I dont see any option for creating that= . But I remember during GSoC, Erom explained those PGA pages through which = admin can do this.

On Sat, Dec 5, 2015 at 2:49 PM, Pamidighantam, Sudhakar= V <spamidig@illinois.edu> wrote:
Pankaj;
is your mesos cluster registered in the PGA as computer resource. Is t= he hello world application catalog entries made. Can you do this in a PGA u= sing the gw56 ( I assume you have admin privileges). If not somebody can do= this for you.=C2=A0

Thanks,
Sudhakar.

On Dec 5, 2015, at 1:26 PM, Pankaj Saha <psaha4@binghamton.edu> wrote:

Hi Supun,=C2=A0
I have pointed to=C2=A0'airavata-server' =3D> 'gw56.iu.xsede.org', to start working with.As you have already provided all the required previl= eges to my new userid, I am able to see all computer resources and applicat= ions in it. So it was not a problem with PGA but Airavata server installati= on in my local machine.

Now can anyone please guide me for launching a hello world application= through this portal so that it can launch a job in a mesos cluster. I know= that I can not use the "gw56.iu.xsede.org" server, as I want to launch a job in my local mesos/marathon cluster. But = if I get some clue how to start with I can possible fix my local Airavata s= erver installation and will try to launch a job in local resource.

Thanks
Pankaj

On Fri, Dec 4, 2015 at 5:41 PM, Supun Nakandala = <supun.na= kandala@gmail.com> wrote:
We haven't done any changes to 0.15 branches recently.= So if it worked for you previously it should still work without any issue.= I suggest that you first try to setup your pga to work with SciGaP hosted = airavata and then try to connect it to your own airavata. But we don't have any active deployments of 0.15. I= n that case you may want to use master branch

On Fri, Dec 4, 2015 at 5:36 PM, Pankaj Saha <psaha4@bingh= amton.edu> wrote:

While running server, no erro= r messages. But while creating projects its throws the error message which = I have posted.

On 04-Dec-2015 4:55 PM, "Supun Nakandala&qu= ot; <supu= n.nakandala@gmail.com> wrote:
do you get any airavata error messages?

On Fri, Dec 4, 2015 at 4:51 PM, Pankaj Saha <psaha4@bingh= amton.edu> wrote:
I am using 15 branch
yes I could not even create a project.

On Fri, Dec 4, 2015 at 4:49 PM, Supun Nakandala = <supun.na= kandala@gmail.com> wrote:
Which Airavata/PGA version are you using?

Looking at your error message I see even the project is also not set.<= /div>

On Fri, Dec 4, 2015 at 12:16 PM, Pankaj Saha <psaha4@bingh= amton.edu> wrote:
Hello Devs,
I have a requirement to launch a job (simple hello world kind of) on m= esos/marathon cluster through Airavata client (PGA preferable ). I have tri= ed installing a local Aiaravata PGA with hosted Airavata server on my machi= ne. Now the problem is with PGA installation, its not allowing me create or explore any project/experiment= .
Its saying:
Required field 'gatewayId' was n= ot present! Struct: createProject_args(gatewayId:null, project:null)

I always used the default =C2=A0gatew= ay-Id which is "default" and the same is mentioned in the server = configuration properties file too. This problem I have never faced earlier = and Airavata installation used to be very simple.

Second=C2=A0challenge=C2=A0is to run = a job through Airavata client, so that it can launch a job in a Mesos clust= er through Marathon. So Airavata should be able to invoke a rest API based = client of Marathon to do the job.
I seek help to resolve the first issu= e and then need to find a way for launching jobs.


Thanks
Pankaj
=C2=A0




--
Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa




--
Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa



--
Thank you
Supun Nakandala
Dept. Computer Science and Engineering
University of Moratuwa





--94eb2c06c59e514659052652ad75--