Mailing-List: contact dev-help@airavata.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airavata.apache.org
MIME-Version: 1.0
In-Reply-To: <CA5219EC-260D-4DF7-B8D0-27EA3A1BDBD8@apache.org>
References: 
 <CAMn8cjhxYJ=Zn7GamiHvjfcdY+gA0Ystmi9_0dXHJv=fGusffg@mail.gmail.com>
 <CA5219EC-260D-4DF7-B8D0-27EA3A1BDBD8@apache.org>
From: Jeffery Kinnison <Jeffery.D.Kinnison.1@nd.edu>
Date: Wed, 27 Apr 2016 16:35:39 -0400
Message-ID: 
 <CAMn8cjgxJmqtQT1Fty6XaF8Bp1apcOeC-b4BVZ=eeEteA=R02Q@mail.gmail.com>
Subject: Re: Planning for In-Situ Application and Resource Monitoring [GSoC
 Project]
To: dev@airavata.apache.org
Content-Type: multipart/alternative; boundary=94eb2c059220a7139c05317d5cce

--94eb2c059220a7139c05317d5cce
Content-Type: text/plain; charset=UTF-8

Thanks Suresh,

I was hoping that I could stick with Python for the meat of the project,
not just because it's the language I'm most comfortable with, but also
thanks to the fact that it's fairly ubiquitous on HPC systems.

I'll take a look at either interfacing the POC with RabbitMQ or converting
it entirely to their Python bindings. If anyone has any alternative
suggestions, they would be much appeciated.

Jeff K.

On Wed, Apr 27, 2016 at 4:20 PM, Suresh Marru <smarru@apache.org> wrote:

> Hi Jeff,
>
> On Apr 27, 2016, at 4:08 PM, Jeffery Kinnison <Jeffery.D.Kinnison.1@nd.edu>
> wrote:
>
> Hi Dev Team,
>
> I'd like to develop a plan for implementing my GSoC project in conjunction
> to getting my development environment up and running. This is my first
> substantial experience with Open Source software development on this scale,
> so thank you in advance for bearing with me.
>
>
> You did great during proposal (hence you have a project), just continue
> the same. At worse you will hear a lot of RTFM which is a common encounter
> in open source. I will let you google for it.
>
> The full project proposal can be found at
> https://cwiki.apache.org/confluence/display/AIRAVATA/GSoC+Proposal+-+In+Situ+Simulation+Analysis+Using+Airavata
>
> The idea is to allow Airavata users to look behind the curtain at jobs
> they are running and enable automatic response to conditions encountered as
> jobs run, both at the system and application level. This will likely
> require a lightweight server to run alongside each job, which will
> communicate with the Airavata server.
>
> I have a prototype for the lightweight server (
> https://github.com/jeffkinnison/simstream) written in Python, however I
> know that Apache software is typically Java-based. The question here is one
> of whether or not the prototype can be rolled into Airavata, or if I need
> to begin looking into Java-based solutions.
>
>
> No, you do not need to port your simstream to Java, infact. Since your
> application demeon will need to run on HPC compute nodes, Java will not be
> a good fit there. I think you should stick to python. For the communication
> with Airavata, one suggestion will be to send a AMQP message which Airavata
> listens to. You can follow this tutorial as a start -
> https://www.rabbitmq.com/tutorials/tutorial-one-python.html. Others may
> have different suggestions.
>
> The other initial question I have is one of how the Airavata server
> submits jobs. From what I can tell, Airavata sends batch scripts to
> connected computing resources, and my thinking right now about how to
> deploy the lightweight server is to add its startup logic to the submit
> scripts. Is this the correct thinking?
>
>
> Yes thats exactly right. As you might see from other discussions, the
> cloud based submissions might not have a batch script, but its fair to
> assume your server will be launched one way or another.
>
>
> Again, thank you for answering these questions, and I'm looking forward to
> working with everyone this summer.
>
>
> Keep them coming.
>
> Suresh
>
>
> Best,
> Jeff K.
>
>
>

--94eb2c059220a7139c05317d5cce
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div>Thanks Suresh,<br><br></div><div>I was hoping th=
at I could stick with Python for the meat of the project, not just because =
it&#39;s the language I&#39;m most comfortable with, but also thanks to the=
 fact that it&#39;s fairly ubiquitous on HPC systems. <br></div><br>I&#39;l=
l take a look at either interfacing the POC with RabbitMQ or converting it =
entirely to their Python bindings. If anyone has any alternative suggestion=
s, they would be much appeciated.<br><br></div>Jeff K.<br></div><div class=
=3D"gmail_extra"><br><div class=3D"gmail_quote">On Wed, Apr 27, 2016 at 4:2=
0 PM, Suresh Marru <span dir=3D"ltr">&lt;<a href=3D"mailto:smarru@apache.or=
g" target=3D"_blank">smarru@apache.org</a>&gt;</span> wrote:<br><blockquote=
 class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli=
d;padding-left:1ex"><div style=3D"word-wrap:break-word">Hi Jeff,<div><br><d=
iv><span class=3D""><blockquote type=3D"cite"><div>On Apr 27, 2016, at 4:08=
 PM, Jeffery Kinnison &lt;<a href=3D"mailto:Jeffery.D.Kinnison.1@nd.edu" ta=
rget=3D"_blank">Jeffery.D.Kinnison.1@nd.edu</a>&gt; wrote:</div><br><div><d=
iv dir=3D"ltr"><div><div><div><div>Hi Dev Team,<br><br></div>I&#39;d like t=
o develop a plan for implementing my GSoC project in conjunction to getting=
 my development environment up and running. This is my first substantial ex=
perience with Open Source software development on this scale, so thank you =
in advance for bearing with me.<br></div></div></div></div></div></blockquo=
te><div><br></div></span>You did great during proposal (hence you have a pr=
oject), just continue the same. At worse you will hear a lot of RTFM which =
is a common encounter in open source. I will let you google for it.=C2=A0</=
div><div><span class=3D""><br><blockquote type=3D"cite"><div><div dir=3D"lt=
r"><div><div>The full project proposal can be found at <a href=3D"https://c=
wiki.apache.org/confluence/display/AIRAVATA/GSoC+Proposal+-+In+Situ+Simulat=
ion+Analysis+Using+Airavata" target=3D"_blank">https://cwiki.apache.org/con=
fluence/display/AIRAVATA/GSoC+Proposal+-+In+Situ+Simulation+Analysis+Using+=
Airavata</a><br><br></div>The idea is to allow Airavata users to look behin=
d the curtain at jobs they are running and enable automatic response to con=
ditions encountered as jobs run, both at the system and application level. =
This will likely require a lightweight server to run alongside each job, wh=
ich will communicate with the Airavata server.<br><br></div><div>I have a p=
rototype for the lightweight server (<a href=3D"https://github.com/jeffkinn=
ison/simstream" target=3D"_blank">https://github.com/jeffkinnison/simstream=
</a>) written in Python, however I know that Apache software is typically J=
ava-based. The question here is one of whether or not the prototype can be =
rolled into Airavata, or if I need to begin looking into Java-based solutio=
ns.<br></div></div></div></blockquote><div><br></div></span><div><div>No, y=
ou do not need to port your=C2=A0simstream to Java, infact. Since your appl=
ication demeon will need to run on HPC compute nodes, Java will not be a go=
od fit there. I think you should stick to python. For the communication wit=
h Airavata, one suggestion will be to send a AMQP message which Airavata li=
stens to. You can follow this tutorial as a start -=C2=A0<a href=3D"https:/=
/www.rabbitmq.com/tutorials/tutorial-one-python.html" target=3D"_blank">htt=
ps://www.rabbitmq.com/tutorials/tutorial-one-python.html</a>. Others may ha=
ve different suggestions.=C2=A0</div><div><br></div></div><span class=3D"">=
<blockquote type=3D"cite"><div><div dir=3D"ltr"><div>The other initial ques=
tion I have is one of how the Airavata server submits jobs. From what I can=
 tell, Airavata sends batch scripts to connected computing resources, and m=
y thinking right now about how to deploy the lightweight server is to add i=
ts startup logic to the submit scripts. Is this the correct thinking?<br></=
div></div></div></blockquote><div><br></div></span><div>Yes thats exactly r=
ight. As you might see from other discussions, the cloud based submissions =
might not have a batch script, but its fair to assume your server will be l=
aunched one way or another.=C2=A0</div><span class=3D""><br><blockquote typ=
e=3D"cite"><div><div dir=3D"ltr"><div><br></div><div>Again, thank you for a=
nswering these questions, and I&#39;m looking forward to working with every=
one this summer.<br></div></div></div></blockquote><div><br></div></span>Ke=
ep them coming.=C2=A0</div><div><br></div><div>Suresh</div><div><br><blockq=
uote type=3D"cite"><div><div dir=3D"ltr"><div><br></div><div>Best,<br></div=
><div>Jeff K.=C2=A0 <br></div></div>
</div></blockquote></div><br></div></div></blockquote></div><br></div>

--94eb2c059220a7139c05317d5cce--