Mailing-List: contact dev-help@airavata.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airavata.apache.org
Received-SPF: Pass (protection.outlook.com: domain of wur.nl designates
 137.224.8.151 as permitted sender) receiver=protection.outlook.com;
 client-ip=137.224.8.151; helo=scomp5307.wurnet.nl;
MIME-Version: 1.0
In-Reply-To: <d447266daddc4edebe13bc3225a3121a@scomp5294.wurnet.nl>
References: 
 <CAJLbdijJ_H2+4kuZwninrjQvTfqSY+A7t39+S-dmrtht724QUA@mail.gmail.com>
 <CABFJMzM4CZ3qNR-4pBxYcXNOWW9U0r15o+HHt0fO41-bHuS09A@mail.gmail.com>
 <CAJvLytpr8Ea=NwExmDva=RODXDNE=KiFrCYzwunp_C54M5mhCQ@mail.gmail.com>
 <EC53CF1A-70E5-402F-ABB9-7F77627F5792@apache.org>
 <d447266daddc4edebe13bc3225a3121a@scomp5294.wurnet.nl>
From: Eric Kuyt <eric.kuijt@wur.nl>
Date: Thu, 25 Feb 2016 09:10:20 +0100
Message-ID: 
 <CAJLbdiiiceQ9WKX9p2tc=drTKvf_GAg6=H0NrOutzhAM6AmGtw@mail.gmail.com>
Subject: Re: Adding a SGE resourceJobManagerType
To: "dev@airavata.apache.org" <dev@airavata.apache.org>
Content-Type: multipart/alternative; boundary="001a1139afa0d4e5cf052c93b714"
SpamDiagnosticOutput: 1:23
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Feb 2016 08:10:41.2995
 (UTC)
X-MS-Exchange-CrossTenant-Id: 27d137e5-761f-4dc1-af88-d26430abb18f
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: 
 TenantId=27d137e5-761f-4dc1-af88-d26430abb18f;Ip=[137.224.8.151];Helo=[scomp5307.wurnet.nl]
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0101MB1728

--001a1139afa0d4e5cf052c93b714
Content-Type: text/plain; charset="UTF-8"

Hey everyone, thanks for all the enthusiastic reactions, I think I can play
around with this tomorrow.

I noticed earlier however that the pbs file generated for univa grid engine
was giving an error code on qsub. Will read in univa today and if it is
supposed tot be backwards compatible.

Eric

On 24 February 2016 at 22:15, Pierce, Marlon <marpierc@iu.edu> wrote:

> Hi Eric, did this work for you?
>
> From: Suresh Marru <smarru@apache.org<mailto:smarru@apache.org>>
> Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" <
> dev@airavata.apache.org<mailto:dev@airavata.apache.org>>
> Date: Wednesday, February 24, 2016 at 10:30 AM
> To: Airavata Dev <dev@airavata.apache.org<mailto:dev@airavata.apache.org>>
> Subject: Re: Adding a SGE resourceJobManagerType
>
> SGE - Sun Grid Engine is the old variant of Univa Grid Engine, so you
> should pick UGE in Airavata and should be good to go. SGE/UGE implement PBS
> protocol so parsing should be similar. Lonestar4 used to have UGE, so email
> parsing should be similar to that.
>
> Suresh
>
>
> On Feb 24, 2016, at 10:17 AM, Shameera Rathnayaka <shameerainfo@gmail.com
> <mailto:shameerainfo@gmail.com>> wrote:
>
> Hi Eric,
>
> There are couple of things you need to do to add a new jobmanager type.
>
> 1. Configure your SGE resource type in gfac yaml config file
> (airavata/modules/configuration/server/src/main/resources/gfac-config.yaml.config).
> As you can see you need to provide an output parser and email parser for
> your SGE, Unless you can't use existing implementations you need to come up
> with your own implementations of these interfaces.
>
> 2. Don't modify thrift generated file, instead add your new
> ResourceJobManagerType to
> airavata/thrift-interface-descriptions/data-models/resource-catalog-models/compute_resource_model.thrift
> file and generate thrift classes.
>
> 3. Come up with SGE JobConfiguration class, and modify
> org.apache.airavata.gfac.impl.Factory#getJobManagerConfiguration method to
> provide it. You should have xslt template for your SGE script file.
>
> let us know if you hit any issue while you following these steps.
>
> Thanks,
> Shameera.
>
> On Wed, Feb 24, 2016 at 9:53 AM Chathuri Wimalasena <kamalasini@gmail.com
> <mailto:kamalasini@gmail.com>> wrote:
> Hi Eric,
>
> To understand what you did, you added a new ResourceJobManagerType in the
> thrift definition as "SGE" ? I assume you generate thrift stubs after
> adding it.  For that, you need to have thrift 0.93 installed in your system.
>
> Thanks..
> Chathuri
>
> On Wed, Feb 24, 2016 at 9:03 AM, Eric Kuyt <eric.kuijt@wur.nl<mailto:
> eric.kuijt@wur.nl>> wrote:
> Hey all, I am playing around with airavata and when I can run jobs on my
> SGE using PBS but the parsing of qsub / qstat fails. So my jobs never leave
> the Executing state.
>
> Now I try to make my own SGEJobConfiguration class.
> Created the class plus outputParser in org.apache.airavata.gfac.impl.job
>
> - add SGE in
> /airavata-gfac-impl/src/main/java/org/apache/airavata/gfac/impl/Factory.java
>
> - add SGE in
> /airavata-data-models/src/main/java/org/apache/airavata/model/appcatalog/computeresource/ResourceJobManagerType.java
>
> - add SGE to
> airavata-php-gateway/app/libraries/Airavata/Model/Job/Types.php in PGA.
>
> What should the next step be? if I Wanted to create my own SGEOutputparser?
> PGA now gives me a Required field 'resourceJobManagerType' is unset! when
> I select SGE in the list.
>
> Thanks,
>
> Eric
>
>
> --
> Central Veterinary Institute of Wageningen UR (CVI)
> Department of Infection Biology
> PO box 65, 8200 AB Lelystad, NL
> Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad
>
> Tel:  +31-(0)320-293391
> Fax: +31-(0)320-238153
> E-mail: eric.kuijt@wur.nl<mailto:eric.kuijt@wur.nl>
> Web: http://www.cvi.wur.nl<http://www.cvi.wur.nl/>
>
> --
> Shameera Rathnayaka
>
>


-- 
Central Veterinary Institute of Wageningen UR (CVI)
Department of Infection Biology
PO box 65, 8200 AB Lelystad, NL
Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad

Tel:  +31-(0)320-293391
Fax: +31-(0)320-238153
E-mail: eric.kuijt@wur.nl
Web: http://www.cvi.wur.nl

--001a1139afa0d4e5cf052c93b714
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:verdana,=
sans-serif;color:#444444">Hey everyone, thanks for all the enthusiastic rea=
ctions, I think I can play around with this tomorrow.=C2=A0</div><div class=
=3D"gmail_default" style=3D"font-family:verdana,sans-serif;color:#444444"><=
br></div><div class=3D"gmail_default" style=3D"font-family:verdana,sans-ser=
if;color:#444444">I noticed earlier however that the pbs file generated for=
 univa grid engine was giving an error code on qsub. Will read in univa tod=
ay and if it is supposed tot be backwards compatible.</div><div class=3D"gm=
ail_default" style=3D"font-family:verdana,sans-serif;color:#444444"><br></d=
iv><div class=3D"gmail_default" style=3D"font-family:verdana,sans-serif;col=
or:#444444">Eric =C2=A0</div></div><div class=3D"gmail_extra"><br><div clas=
s=3D"gmail_quote">On 24 February 2016 at 22:15, Pierce, Marlon <span dir=3D=
"ltr">&lt;<a href=3D"mailto:marpierc@iu.edu" target=3D"_blank">marpierc@iu.=
edu</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Eric, did th=
is work for you?<br>
<br>
From: Suresh Marru &lt;<a href=3D"mailto:smarru@apache.org">smarru@apache.o=
rg</a>&lt;mailto:<a href=3D"mailto:smarru@apache.org">smarru@apache.org</a>=
&gt;&gt;<br>
<span class=3D"">Reply-To: &quot;<a href=3D"mailto:dev@airavata.apache.org"=
>dev@airavata.apache.org</a>&lt;mailto:<a href=3D"mailto:dev@airavata.apach=
e.org">dev@airavata.apache.org</a>&gt;&quot; &lt;<a href=3D"mailto:dev@aira=
vata.apache.org">dev@airavata.apache.org</a>&lt;mailto:<a href=3D"mailto:de=
v@airavata.apache.org">dev@airavata.apache.org</a>&gt;&gt;<br>
</span>Date: Wednesday, February 24, 2016 at 10:30 AM<br>
To: Airavata Dev &lt;<a href=3D"mailto:dev@airavata.apache.org">dev@airavat=
a.apache.org</a>&lt;mailto:<a href=3D"mailto:dev@airavata.apache.org">dev@a=
iravata.apache.org</a>&gt;&gt;<br>
<span class=3D"im HOEnZb">Subject: Re: Adding a SGE resourceJobManagerType<=
br>
<br>
</span><span class=3D"im HOEnZb">SGE - Sun Grid Engine is the old variant o=
f Univa Grid Engine, so you should pick UGE in Airavata and should be good =
to go. SGE/UGE implement PBS protocol so parsing should be similar. Lonesta=
r4 used to have UGE, so email parsing should be similar to that.<br>
<br>
Suresh<br>
<br>
<br>
On Feb 24, 2016, at 10:17 AM, Shameera Rathnayaka &lt;<a href=3D"mailto:sha=
meerainfo@gmail.com">shameerainfo@gmail.com</a>&lt;mailto:<a href=3D"mailto=
:shameerainfo@gmail.com">shameerainfo@gmail.com</a>&gt;&gt; wrote:<br>
<br>
Hi Eric,<br>
<br>
There are couple of things you need to do to add a new jobmanager type.<br>
<br>
1. Configure your SGE resource type in gfac yaml config file (airavata/modu=
les/configuration/server/src/main/resources/gfac-config.yaml.config).=C2=A0=
 As you can see you need to provide an output parser and email parser for y=
our SGE, Unless you can&#39;t use existing implementations you need to come=
 up with your own implementations of these interfaces.<br>
<br>
2. Don&#39;t modify thrift generated file, instead add your new ResourceJob=
ManagerType to airavata/thrift-interface-descriptions/data-models/resource-=
catalog-models/compute_resource_model.thrift=C2=A0 file and generate thrift=
 classes.<br>
<br>
3. Come up with SGE JobConfiguration class, and modify org.apache.airavata.=
gfac.impl.Factory#getJobManagerConfiguration method to provide it. You shou=
ld have xslt template for your SGE script file.<br>
<br>
let us know if you hit any issue while you following these steps.<br>
<br>
Thanks,<br>
Shameera.<br>
<br>
On Wed, Feb 24, 2016 at 9:53 AM Chathuri Wimalasena &lt;<a href=3D"mailto:k=
amalasini@gmail.com">kamalasini@gmail.com</a>&lt;mailto:<a href=3D"mailto:k=
amalasini@gmail.com">kamalasini@gmail.com</a>&gt;&gt; wrote:<br>
Hi Eric,<br>
<br>
To understand what you did, you added a new ResourceJobManagerType in the t=
hrift definition as &quot;SGE&quot; ? I assume you generate thrift stubs af=
ter adding it.=C2=A0 For that, you need to have thrift 0.93 installed in yo=
ur system.<br>
<br>
Thanks..<br>
Chathuri<br>
<br>
On Wed, Feb 24, 2016 at 9:03 AM, Eric Kuyt &lt;<a href=3D"mailto:eric.kuijt=
@wur.nl">eric.kuijt@wur.nl</a>&lt;mailto:<a href=3D"mailto:eric.kuijt@wur.n=
l">eric.kuijt@wur.nl</a>&gt;&gt; wrote:<br>
</span><span class=3D"im HOEnZb">Hey all, I am playing around with airavata=
 and when I can run jobs on my SGE using PBS but the parsing of qsub / qsta=
t fails. So my jobs never leave the Executing state.<br>
<br>
Now I try to make my own SGEJobConfiguration class.<br>
Created the class plus outputParser in org.apache.airavata.gfac.impl.job<br=
>
<br>
- add SGE in /airavata-gfac-impl/src/main/java/org/apache/airavata/gfac/imp=
l/Factory.java<br>
<br>
- add SGE in /airavata-data-models/src/main/java/org/apache/airavata/model/=
appcatalog/computeresource/ResourceJobManagerType.java<br>
<br>
- add SGE to airavata-php-gateway/app/libraries/Airavata/Model/Job/Types.ph=
p in PGA.<br>
<br>
What should the next step be? if I Wanted to create my own SGEOutputparser?=
<br>
PGA now gives me a Required field &#39;resourceJobManagerType&#39; is unset=
! when I select SGE in the list.<br>
<br>
Thanks,<br>
<br>
Eric<br>
<br>
<br>
--<br>
Central Veterinary Institute of Wageningen UR (CVI)<br>
Department of Infection Biology<br>
PO box 65, 8200 AB Lelystad, NL<br>
Visiting address: ASG, Edelhertweg 15, 8219 PH Lelystad<br>
<br>
Tel:=C2=A0 <a href=3D"tel:%2B31-%280%29320-293391" value=3D"+31320293391">+=
31-(0)320-293391</a><br>
Fax: <a href=3D"tel:%2B31-%280%29320-238153" value=3D"+31320238153">+31-(0)=
320-238153</a><br>
</span><div class=3D"HOEnZb"><div class=3D"h5">E-mail: <a href=3D"mailto:er=
ic.kuijt@wur.nl">eric.kuijt@wur.nl</a>&lt;mailto:<a href=3D"mailto:eric.kui=
jt@wur.nl">eric.kuijt@wur.nl</a>&gt;<br>
Web: <a href=3D"http://www.cvi.wur.nl" rel=3D"noreferrer" target=3D"_blank"=
>http://www.cvi.wur.nl</a>&lt;<a href=3D"http://www.cvi.wur.nl/" rel=3D"nor=
eferrer" target=3D"_blank">http://www.cvi.wur.nl/</a>&gt;<br>
<br>
--<br>
Shameera Rathnayaka<br>
<br>
</div></div></blockquote></div><br><br clear=3D"all"><div><br></div>-- <br>=
<div class=3D"gmail_signature"><div dir=3D"ltr">Central Veterinary Institut=
e of Wageningen UR (CVI)<br>Department of Infection Biology<br>PO box 65, 8=
200 AB Lelystad, NL<br>Visiting address: ASG, Edelhertweg 15, 8219 PH Lelys=
tad<br>=C2=A0<br>Tel:=C2=A0 <a value=3D"+31320293391">+31-(0)320-293391</a>=
<br>Fax: <a value=3D"+31320238153">+31-(0)320-238153</a><br>E-mail: <a href=
=3D"mailto:eric.kuijt@wur.nl" target=3D"_blank">eric.kuijt@wur.nl</a><br>We=
b:<a href=3D"http://www.cvi.wur.nl" target=3D"_blank"> http://www.cvi.wur.n=
l</a></div></div>
</div>

--001a1139afa0d4e5cf052c93b714--