airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shameera Rathnayaka <shameerai...@gmail.com>
Subject Re: Airavata GSoC 2013 Master Project
Date Fri, 03 May 2013 05:52:44 GMT
Hi Suresh et al.

When writing the proposal, I was thinking a better approach to implement
Airavata Server API to use with web UI. I have come across two possible
ways to do this.

1. Implement existing Airavata Server API to generate the JSON messages.
2. Write a new Client API with JavaScript to generate and handle JSON
messages .

In my opinion , Option two would be the best case here. I have added this
parts to my proposal as well and it is now completed. There are few
disadvantage with option one, i have explained it there, please go through
it. And i would like to know about your idea on this.If you have other
alternative ways to do this we can discuss and select the best one.

Thanks,
Shameera.


On Thu, May 2, 2013 at 10:22 PM, Subho Banerjee <subs.zero@gmail.com> wrote:

> Hi Suresh,
> I am in the process of writing the proposal... I should be done by tonight.
> Will post it on the mailing list once I am done.
>
> Cheers,
> Subho
>
>
> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <smarru@apache.org> wrote:
>
> > Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
> >
> > Suresh
> >
> > On May 1, 2013, at 5:13 PM, Suresh Marru <smarru@apache.org> wrote:
> >
> > > Hi Vijayendra,
> > >
> > > As you can see from Shameera's proposal, he proposed a JSON conversion
> > in front of WS Messenger. Also Danuska has been proposing for the AMQP
> and
> > idea and deliberating its advantages. So given all these, I would suggest
> > for you to keep focused on the UI aspects of the monitoring and write
> into
> > your proposal a plan for determining a good strategy for the plumbing to
> > WS-Eventing based existing system. You can have the concrete deliverables
> > of new UI to change colors based on executions (as it currently does in
> > XBaya), double click and show error messages and so forth. And keep it
> > exploratory for the actually messaging format.
> > >
> > > I do not have any opinion on the libraries you mentioned, but yaa a
> ajax
> > based pub system might be the right way to go. Pending the content format
> > (JSON or WS-Eventing or JMS or AMQP or something else)
> > >
> > > Suresh
> > >
> > > On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> > vijayendra.sdm@gmail.com> wrote:
> > >
> > >> Hi Suresh
> > >>
> > >> I am writing proposal for monitoring tool . The monitoring tool is
> > based on
> > >> pub-sub model (ws-messenger).
> > >> While writing proposal , I have to back it by technical stuff that
> tells
> > >> how can we achieve our purpose.
> > >> As this monitoring tool is supposed to be a web based , and we are
> > thinking
> > >> in the lines of
> > >> developing it in javascript.
> > >> I was looking into javascript libraries that can we used with
> > ws-messenger
> > >> in the monitoring module.
> > >> Please correct me if I am wrong.
> > >> I came across some of the libraries
> > >>
> > >>  - jQuery custom
> > >> events<
> > http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> > >>  - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> > >>  - PubSubJS <https://github.com/mroderick/PubSubJS>
> > >>  - js-signals <http://millermedeiros.github.com/js-signals/>
> > >>
> > >> please tell me am I thinking in right direction?
> > >>
> > >> Regards
> > >> Vijayendra
> > >>
> > >>
> > >>
> > >>
> > >> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <smarru@apache.org>
> wrote:
> > >>
> > >>> Hi Shameera,
> > >>>
> > >>> This is great, I appreciate you sharing it, I realize this is still
> > >>> working document, but I want other students to start seeing it and
> > model
> > >>> their proposals in a similar way.
> > >>>
> > >>> Airavata Mentors,
> > >>>
> > >>> Please provide feedback directly on the melange site and uncheck the
> > >>> "private" box when you comment.
> > >>>
> > >>> Suresh
> > >>>
> > >>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> > shameerainfo@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> Hi Suresh and All,
> > >>>>
> > >>>> Of course I am very much happy to share my proposal with everybody,
> > >>>> actually i was going to update this thread with the melange link
in
> > few
> > >>>> hours once i have done writing all the sections in the proposal.
I
> > >>> haven't
> > >>>> yet added the milestone plan into it and now working on it.
> > >>>>
> > >>>> The sub area i am going to work from the Master project  is '
> > >>> Implementing
> > >>>> a JSON interface to Airavata Client side and Registry component'.
> > Here is
> > >>>> the link
> > >>>>
> > >>>
> >
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> > >>>> .
> > >>>>
> > >>>>
> > >>>> Please note that i haven't completed everything in this and still
> > doing
> > >>>> modifications .Therefore proposal content may be changed bit, need
> to
> > add
> > >>>> more technical details of the approach which explains it well.
> > >>>>
> > >>>> I would like to know the feedback from all of you regarding the
> > proposal
> > >>>> and will be modifying it if there is anything to be done. Also
> please
> > >>>> contact me if you need any help and i am very much willing to share
> my
> > >>>> thoughts with all.
> > >>>>
> > >>>> Thanks!
> > >>>> Shameera
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <smarru@apache.org>
> > wrote:
> > >>>>
> > >>>>> Hi Shameera,
> > >>>>>
> > >>>>> Excellent proposal, great job. Would you mind to make  your
> proposal
> > >>>>> public and post the link here? Your proposal should help others
to
> > look
> > >>> at
> > >>>>> it and learn from.
> > >>>>>
> > >>>>> Again I emphasize to all students, please don't feel you will
be
> > >>> competing
> > >>>>> with each others. If all of you write good proposals, there
is a
> good
> > >>>>> chance all of you will be selected. But without a good proposal,
we
> > >>> cannot
> > >>>>> help.
> > >>>>>
> > >>>>> Suresh
> > >>>>>
> > >>>>>
> > >>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> > >>> shameerainfo@gmail.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> Yes it is not easy to solve all problems, But defining
our own
> > standard
> > >>>>> or
> > >>>>>> adhere to any standard
> > >>>>>> provided by third party library will solve the problem
to some
> > extend.
> > >>>>>>
> > >>>>>> Here i see two possible approaches,
> > >>>>>>
> > >>>>>> 1. Use existing third party library(we can find which is
best)
> > adhere
> > >>> to
> > >>>>> it
> > >>>>>> standard and see how we change the
> > >>>>>> backend to be inline with it.
> > >>>>>>
> > >>>>>> 2. Use our own convention with help of XMLSchema (The way
i
> > suggest).
> > >>>>>>
> > >>>>>> As Suresh mentioned we can do a POC with both approaches
to
> compare
> > >>>>>> performance
> > >>>>>> and changes need to be done in server side. Then select
the best
> > one.
> > >>>>>>
> > >>>>>> Another question was, can we works with graph data in JSON
format.
> > >>>>>> There are few JS graph framworks[1] which provide that
> > functionality.
> > >>>>>> we can use one of them to show airavata monitoring data
as graphs
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> Shameera.
> > >>>>>>
> > >>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3
<
> http://d3js.org/>
> > ,
> > >>>>>> Processing.js <http://processingjs.org> , Sencha
> > >>>>>> Charts<http://www.sencha.com/products/extjs/>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <smarru@apache.org>
> > >>> wrote:
> > >>>>>>
> > >>>>>>> Hi Vijeyandra,
> > >>>>>>>
> > >>>>>>> Airavata Messaging is based on a pub-sub model and
the events
> > >>> themselves
> > >>>>>>> are xml (WS-Eventing [1]).
> > >>>>>>>
> > >>>>>>> The Messenger paper [2] should give you more information.
> > >>>>>>>
> > >>>>>>> Hi All (Especially those at WS02):
> > >>>>>>>
> > >>>>>>> Here is an old effort from a Morotuwa undergrad project,
you may
> > want
> > >>> to
> > >>>>>>> read through these papers and chat with the authors
to get
> > >>> experiences:
> > >>>>>>>
> > >>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> > >>>>>>>
> > http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> > >>>>>>>
> > http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> > >>>>>>> http://mooshabaya.wikidot.com/
> > >>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>
> >
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> > >>>>>>>
> > >>>>>>> Suresh
> > >>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> > >>>>>>> [2] -
> > >>>>>>>
> > >>>>>
> > >>>
> > http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> > >>>>>>>
> > >>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit
<
> > >>>>>>> vijayendra.sdm@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> Hi Suresh
> > >>>>>>>>
> > >>>>>>>> I wanted to know more about the monitoring tool
.
> > >>>>>>>> Currently from where does the monitoring tool gets
data . Is it
> > from
> > >>>>>>>> workflow interpreter ? or Is it from the WS Messenger
( that
> might
> > >>>>>>> continuously
> > >>>>>>>> send messages to monitoring tool as to tell how
much is the
> > progress
> > >>>>>>>> and what are the variables getting changed) ?
> > >>>>>>>>
> > >>>>>>>> Again the how is the data being exchanged. I guess
it must be
> xml
> > ?
> > >>>>>>>> It must be one way data exchange . I mean the data
is TO the
> > >>>>>>>> monitoring module.
> > >>>>>>>> Then monitoring Tool  is sending back this
> > >>>>>>>> data to Xbaya for displaying to the user ? Please
correct me if
> I
> > am
> > >>>>>>> wrong
> > >>>>>>>>
> > >>>>>>>> I have downloaded the source code from the trunk
. can you
> please
> > >>> point
> > >>>>>>>> me which part of code should I be code at for this
module.
> > >>>>>>>>
> > >>>>>>>> Regards
> > >>>>>>>> Vijayendra
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit
<
> > >>>>>>> vijayendra.sdm@gmail.com> wrote:
> > >>>>>>>> Hi
> > >>>>>>>>
> > >>>>>>>> What i am suggesting is, we send the JSON message
directly to
> > >>> Airavata
> > >>>>>>>> Backend(or Registry)
> > >>>>>>>> When the message gets build after the transport
phase, convert
> > JSON
> > >>>>>>> message
> > >>>>>>>> to SOAP(XML).
> > >>>>>>>> From that point message will treated as SOAP message.
> > >>>>>>>>
> > >>>>>>>> If we look at the JSON <--> XML conversion
there are set of
> third
> > >>> party
> > >>>>>>>> libraries we
> > >>>>>>>> can use for. But before selecting a one we need
to think about
> > >>> problems
> > >>>>>>>> having
> > >>>>>>>>
> > >>>>>>>> with JSON <--> XML and how these libraries
handle those issues.
> > >>> Because
> > >>>>>>> we
> > >>>>>>>> need a robust
> > >>>>>>>> way to do this conversions.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Shameera what you are suggesting is sending the
JSON message
> > directly
> > >>>>> to
> > >>>>>>> Registry.
> > >>>>>>>> when the message gets built after the transport
phase , convert
> > it to
> > >>>>>>> SOAP .
> > >>>>>>>>
> > >>>>>>>> If you are suggesting Registry will have JSON data
instead of
> > WSDL ,
> > >>>>>>> Then this might
> > >>>>>>>> complicate the things  for us .
> > >>>>>>>> The workflow interpreter needs wsdl(xml) to interpret
the
> > workflows
> > >>> and
> > >>>>>>> for other details .
> > >>>>>>>> Which means we might again have to do some changes
with workflow
> > >>>>>>> interpretor .
> > >>>>>>>> Yesterday from what I heard in discussion is that
, they do not
> > want
> > >>> to
> > >>>>>>> mess with workflow
> > >>>>>>>> interpreter atleast for GSOC projects.
> > >>>>>>>>
> > >>>>>>>> What I want to suggest is , why carry the  JSON
data till
> > Regisrty .
> > >>>>>>> Build a interface
> > >>>>>>>> before (Apache server API) where we  can do the
necessary
> > conversion
> > >>>>>>> (JSON  to  SOAP).
> > >>>>>>>> In this way we can avoid messing up with Airavata
server as a
> > whole.
> > >>>>>>>> Client ( using a we browser) is interacting with
JSON (web
> > service)
> > >>> but
> > >>>>>>> the Apache server
> > >>>>>>>> is interacting with SOAP.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Secondly yesterday Suresh was speaking about validating
the
> > >>> connections
> > >>>>>>> of the workflow.
> > >>>>>>>> for example , the workflow is expecting a file
as input
> > >>>>>>>> but user is giving a sting  or int .
> > >>>>>>>>
> > >>>>>>>> Here what I suggest is , while creating wsdl in
the registry
> for a
> > >>>>>>> particular
> > >>>>>>>> workflow , we can add extra information in the
form of
> > >>>>>>>> annotation as  the kind of input/ output the workflow
is
> > accepting.
> > >>>>>>>> Then we will be able to check these against users
entry during
> > >>>>> execution.
> > >>>>>>>> Please correct me if I am wrong.
> > >>>>>>>>
> > >>>>>>>> Regards
> > >>>>>>>> Vijayendra
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee
<
> > subs.zero@gmail.com
> > >>>>
> > >>>>>>> wrote:
> > >>>>>>>> Well exactly, as long as you can define standard
way of
> > >>> communication.
> > >>>>>>> That
> > >>>>>>>> is, you can define in advance what should be a
string, array and
> > what
> > >>>>>>>> should be a integer etc. We have no problem.
> > >>>>>>>>
> > >>>>>>>> So, when you look at problems, with JSON <->
XML or the other
> way
> > >>>>> round,
> > >>>>>>>> they talk of the very general case (where you no
nothing about
> the
> > >>> data
> > >>>>>>> you
> > >>>>>>>> are converting other than it is valid XML/JSON).
There are a
> > myriad
> > >>> of
> > >>>>>>>> problems in that case, which you pointed out.
> > >>>>>>>>
> > >>>>>>>> But when there is standard, there is only one way
of doing
> things,
> > >>> and
> > >>>>>>> not
> > >>>>>>>> several. I think that is the way forward. So what
I am proposing
> > is
> > >>>>> maybe
> > >>>>>>>> we all discuss and define this standard within
the first week of
> > GSoC
> > >>>>>>>> starting and then actually move into coding. So
as long as we
> work
> > >>> with
> > >>>>>>> the
> > >>>>>>>> presumption that this will be done, we really dont
have to
> worry a
> > >>> lot
> > >>>>>>>> about this.
> > >>>>>>>>
> > >>>>>>>> Cheers,
> > >>>>>>>> Subho.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka
<
> > >>>>>>>> shameerainfo@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi,
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee
<
> > >>> subs.zero@gmail.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Some of these problems are very specific
to what the XML is
> > >>>>>>>>> representing,
> > >>>>>>>>>> it might not be an actual problem in Airavata,
> > >>>>>>>>>> maybe some one more experienced with the
codebase can point
> this
> > >>> out.
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> All issues pointed out in the paper is not
directly valid to
> our
> > >>>>>>>>> conversion, I didn't list the issues actually
need to address
> in
> > >>> this
> > >>>>>>> case
> > >>>>>>>>> because thought it is worth to read that introduction
part
> which
> > >>>>>>> explain
> > >>>>>>>>> the all the issues we have with this conversion
and give us a
> > solid
> > >>>>>>>>> background of that.
> > >>>>>>>>>
> > >>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing,
Character sets
> --
> > I
> > >>>>>>>>> really
> > >>>>>>>>>> dont see these as problems, as long as
you can agree that all
> > >>>>>>> parts of
> > >>>>>>>>>> airavata will treat the JSON in a standard
(probably we have
> to
> > >>>>>>> define
> > >>>>>>>>>> this) way.
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> The issue with JSON array only comes when we
try to convert XML
> > to
> > >>>>>>> JSON not
> > >>>>>>>>> the other way. If we map with JSON, inputparameters
and
> > >>>>>>> outputparameters in
> > >>>>>>>>> the ServiceDescription.xsd will map with JSON
Arrays. Therefore
> > we
> > >>>>>>> need to
> > >>>>>>>>> solve this issue.
> > >>>>>>>>>
> > >>>>>>>>> JSON XML JSON
> > >>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs>
 -->
> > >>> {"inputs":["test"]}
> > >>>>>>> //
> > >>>>>>>>> correct one
> > >>>>>>>>>                        --> {"inputs":"test"}
    // incorrect
> one
> > >>>>>>>>>
> > >>>>>>>>> 2. Namespaces, Processing Instructions -- Is
this required?
> > >>>>>>>>>
> > >>>>>>>>>> Are separate namespaces used in Airavata?
Only place I can see
> > >>>>>>> this
> > >>>>>>>>>> being
> > >>>>>>>>>> used is probably in the WSDL, but if we
can agree on another
> way
> > >>>>>>>>>> of communicating registered applications'
I/O parameters to
> the
> > >>>>>>> front
> > >>>>>>>>>> end
> > >>>>>>>>>> (JSON based), then maybe we can work around
this (minor)
> > problem.
> > >>>>>>> Are
> > >>>>>>>>>> custom processing instructions to the Xbaya
XML parse even
> used?
> > >>>>>>>>>> 3. Attributes -- Again, this can be fixed
easily
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> Yes,attributes convertion will not be a big
issues we can solve
> > it.
> > >>> As
> > >>>>>>>>> Lahiru mentioned in Hangout session namesapce
handling is not a
> > big
> > >>>>>>> issue
> > >>>>>>>>> with Airavata.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> <array name="abc">
> > >>>>>>>>>>  <element>1</element>
> > >>>>>>>>>>  <element>2</element>
> > >>>>>>>>>>  <element>3</element>
> > >>>>>>>>>>  <element>4</element>
> > >>>>>>>>>> </array>
> > >>>>>>>>>>
> > >>>>>>>>>> Can become
> > >>>>>>>>>>
> > >>>>>>>>>> {
> > >>>>>>>>>>
> > >>>>>>>>>> abc : ['1', '2', '3', '4']
> > >>>>>>>>>>
> > >>>>>>>>>> }
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> With this example it show us we need to change
the XML message
> > >>> format
> > >>>>>>> of
> > >>>>>>>>> server side, which require to change the all
schemas, If we are
> > >>> going
> > >>>>>>> to
> > >>>>>>>>> change the schemas then we need to change the
way it process it
> > in
> > >>>>>>> Ariavara
> > >>>>>>>>> core. We have dropped our initial major requirement,
which is
> > keep
> > >>> the
> > >>>>>>>>> Airavata Server side as it is.
> > >>>>>>>>>
> > >>>>>>>>> with this conversion we only deal with json
strings, yes we can
> > send
> > >>>>>>> JSON
> > >>>>>>>>> request with other formats supported by JSON
like boolen, null,
> > >>> Number
> > >>>>>>>>> etc.. But there is no way to get the same JSON
from XML as XML
> > only
> > >>>>>>> deal
> > >>>>>>>>> only with Strings. I think it is good if we
can consume a this
> > >>>>> features
> > >>>>>>>>> with JSON.
> > >>>>>>>>>
> > >>>>>>>>> let say i need to send a integer or float to
the server using
> > JSON
> > >>>>> then
> > >>>>>>>>> proper way is to send {"<name>":123.45}
this will works fine
> but
> > >>>>>>> problem is
> > >>>>>>>>> how we get the same output ?
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>> Shameera.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Cheers,
> > >>>>>>>>>> Subho.
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Best Regards,
> > >>>>>>>>> Shameera Rathnayaka.
> > >>>>>>>>>
> > >>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Best Regards,
> > >>>>>> Shameera Rathnayaka.
> > >>>>>>
> > >>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Best Regards,
> > >>>> Shameera Rathnayaka.
> > >>>>
> > >>>> email: shameera AT apache.org , shameerainfo AT gmail.com
> > >>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>
> > >>>
> > >
> >
> >
>



-- 
Best Regards,
Shameera Rathnayaka.

email: shameera AT apache.org , shameerainfo AT gmail.com
Blog : http://shameerarathnayaka.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message