airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vijayendra Grampurohit <vijayendra....@gmail.com>
Subject Re: Airavata GSoC 2013 Master Project
Date Wed, 01 May 2013 20:13:48 GMT
Hi Suresh

I am writing proposal for monitoring tool . The monitoring tool is based on
pub-sub model (ws-messenger).
While writing proposal , I have to back it by technical stuff that tells
how can we achieve our purpose.
As this monitoring tool is supposed to be a web based , and we are thinking
in the lines of
developing it in javascript.
I was looking into javascript libraries that can we used with ws-messenger
in the monitoring module.
Please correct me if I am wrong.
I came across some of the libraries

   - jQuery custom
events<http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
   - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
   - PubSubJS <https://github.com/mroderick/PubSubJS>
   - js-signals <http://millermedeiros.github.com/js-signals/>

please tell me am I thinking in right direction?

Regards
Vijayendra




On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <smarru@apache.org> wrote:

> Hi Shameera,
>
> This is great, I appreciate you sharing it, I realize this is still
> working document, but I want other students to start seeing it and model
> their proposals in a similar way.
>
> Airavata Mentors,
>
> Please provide feedback directly on the melange site and uncheck the
> "private" box when you comment.
>
> Suresh
>
> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <shameerainfo@gmail.com>
> wrote:
>
> > Hi Suresh and All,
> >
> > Of course I am very much happy to share my proposal with everybody,
> > actually i was going to update this thread with the melange link in few
> > hours once i have done writing all the sections in the proposal. I
> haven't
> > yet added the milestone plan into it and now working on it.
> >
> > The sub area i am going to work from the Master project  is '
> Implementing
> > a JSON interface to Airavata Client side and Registry component'. Here is
> > the link
> >
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> > .
> >
> >
> > Please note that i haven't completed everything in this and still doing
> > modifications .Therefore proposal content may be changed bit, need to add
> > more technical details of the approach which explains it well.
> >
> > I would like to know the feedback from all of you regarding the proposal
> > and will be modifying it if there is anything to be done. Also please
> > contact me if you need any help and i am very much willing to share my
> > thoughts with all.
> >
> > Thanks!
> > Shameera
> >
> >
> >
> > On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <smarru@apache.org> wrote:
> >
> >> Hi Shameera,
> >>
> >> Excellent proposal, great job. Would you mind to make  your proposal
> >> public and post the link here? Your proposal should help others to look
> at
> >> it and learn from.
> >>
> >> Again I emphasize to all students, please don't feel you will be
> competing
> >> with each others. If all of you write good proposals, there is a good
> >> chance all of you will be selected. But without a good proposal, we
> cannot
> >> help.
> >>
> >> Suresh
> >>
> >>
> >> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> shameerainfo@gmail.com>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> Yes it is not easy to solve all problems, But defining our own standard
> >> or
> >>> adhere to any standard
> >>> provided by third party library will solve the problem to some extend.
> >>>
> >>> Here i see two possible approaches,
> >>>
> >>> 1. Use existing third party library(we can find which is best) adhere
> to
> >> it
> >>> standard and see how we change the
> >>>   backend to be inline with it.
> >>>
> >>> 2. Use our own convention with help of XMLSchema (The way i suggest).
> >>>
> >>> As Suresh mentioned we can do a POC with both approaches to compare
> >>> performance
> >>> and changes need to be done in server side. Then select the best one.
> >>>
> >>> Another question was, can we works with graph data in JSON format.
> >>> There are few JS graph framworks[1] which provide that functionality.
> >>> we can use one of them to show airavata monitoring data as graphs
> >>>
> >>> Thanks,
> >>> Shameera.
> >>>
> >>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/>
,
> >>> Processing.js <http://processingjs.org> , Sencha
> >>> Charts<http://www.sencha.com/products/extjs/>
> >>>
> >>>
> >>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <smarru@apache.org>
> wrote:
> >>>
> >>>> Hi Vijeyandra,
> >>>>
> >>>> Airavata Messaging is based on a pub-sub model and the events
> themselves
> >>>> are xml (WS-Eventing [1]).
> >>>>
> >>>> The Messenger paper [2] should give you more information.
> >>>>
> >>>> Hi All (Especially those at WS02):
> >>>>
> >>>> Here is an old effort from a Morotuwa undergrad project, you may want
> to
> >>>> read through these papers and chat with the authors to get
> experiences:
> >>>>
> >>>> http://dl.acm.org/citation.cfm?id=1890807
> >>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> >>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> >>>> http://mooshabaya.wikidot.com/
> >>>>
> >>>>
> >>
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> >>>>
> >>>> Suresh
> >>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> >>>> [2] -
> >>>>
> >>
> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> >>>>
> >>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> >>>> vijayendra.sdm@gmail.com> wrote:
> >>>>
> >>>>> Hi Suresh
> >>>>>
> >>>>> I wanted to know more about the monitoring tool .
> >>>>> Currently from where does the monitoring tool gets data . Is it
from
> >>>>> workflow interpreter ? or Is it from the WS Messenger ( that might
> >>>> continuously
> >>>>> send messages to monitoring tool as to tell how much is the progress
> >>>>> and what are the variables getting changed) ?
> >>>>>
> >>>>> Again the how is the data being exchanged. I guess it must be xml
?
> >>>>> It must be one way data exchange . I mean the data is TO the
> >>>>> monitoring module.
> >>>>> Then monitoring Tool  is sending back this
> >>>>> data to Xbaya for displaying to the user ? Please correct me if
I am
> >>>> wrong
> >>>>>
> >>>>> I have downloaded the source code from the trunk . can you please
> point
> >>>>> me which part of code should I be code at for this module.
> >>>>>
> >>>>> Regards
> >>>>> Vijayendra
> >>>>>
> >>>>>
> >>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> >>>> vijayendra.sdm@gmail.com> wrote:
> >>>>> Hi
> >>>>>
> >>>>> What i am suggesting is, we send the JSON message directly to
> Airavata
> >>>>> Backend(or Registry)
> >>>>> When the message gets build after the transport phase, convert JSON
> >>>> message
> >>>>> to SOAP(XML).
> >>>>> From that point message will treated as SOAP message.
> >>>>>
> >>>>> If we look at the JSON <--> XML conversion there are set of
third
> party
> >>>>> libraries we
> >>>>> can use for. But before selecting a one we need to think about
> problems
> >>>>> having
> >>>>>
> >>>>> with JSON <--> XML and how these libraries handle those issues.
> Because
> >>>> we
> >>>>> need a robust
> >>>>> way to do this conversions.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Shameera what you are suggesting is sending the JSON message directly
> >> to
> >>>> Registry.
> >>>>> when the message gets built after the transport phase , convert
it to
> >>>> SOAP .
> >>>>>
> >>>>> If you are suggesting Registry will have JSON data instead of WSDL
,
> >>>> Then this might
> >>>>> complicate the things  for us .
> >>>>> The workflow interpreter needs wsdl(xml) to interpret the workflows
> and
> >>>> for other details .
> >>>>> Which means we might again have to do some changes with workflow
> >>>> interpretor .
> >>>>> Yesterday from what I heard in discussion is that , they do not
want
> to
> >>>> mess with workflow
> >>>>> interpreter atleast for GSOC projects.
> >>>>>
> >>>>> What I want to suggest is , why carry the  JSON data till Regisrty
.
> >>>> Build a interface
> >>>>> before (Apache server API) where we  can do the necessary conversion
> >>>> (JSON  to  SOAP).
> >>>>> In this way we can avoid messing up with Airavata server as a whole.
> >>>>> Client ( using a we browser) is interacting with JSON (web service)
> but
> >>>> the Apache server
> >>>>> is interacting with SOAP.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Secondly yesterday Suresh was speaking about validating the
> connections
> >>>> of the workflow.
> >>>>> for example , the workflow is expecting a file as input
> >>>>> but user is giving a sting  or int .
> >>>>>
> >>>>> Here what I suggest is , while creating wsdl in the registry for
a
> >>>> particular
> >>>>> workflow , we can add extra information in the form of
> >>>>> annotation as  the kind of input/ output the workflow is accepting.
> >>>>> Then we will be able to check these against users entry during
> >> execution.
> >>>>> Please correct me if I am wrong.
> >>>>>
> >>>>> Regards
> >>>>> Vijayendra
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <subs.zero@gmail.com
> >
> >>>> wrote:
> >>>>> Well exactly, as long as you can define standard way of
> communication.
> >>>> That
> >>>>> is, you can define in advance what should be a string, array and
what
> >>>>> should be a integer etc. We have no problem.
> >>>>>
> >>>>> So, when you look at problems, with JSON <-> XML or the other
way
> >> round,
> >>>>> they talk of the very general case (where you no nothing about the
> data
> >>>> you
> >>>>> are converting other than it is valid XML/JSON). There are a myriad
> of
> >>>>> problems in that case, which you pointed out.
> >>>>>
> >>>>> But when there is standard, there is only one way of doing things,
> and
> >>>> not
> >>>>> several. I think that is the way forward. So what I am proposing
is
> >> maybe
> >>>>> we all discuss and define this standard within the first week of
GSoC
> >>>>> starting and then actually move into coding. So as long as we work
> with
> >>>> the
> >>>>> presumption that this will be done, we really dont have to worry
a
> lot
> >>>>> about this.
> >>>>>
> >>>>> Cheers,
> >>>>> Subho.
> >>>>>
> >>>>>
> >>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> >>>>> shameerainfo@gmail.com> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> subs.zero@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Some of these problems are very specific to what the XML
is
> >>>>>> representing,
> >>>>>>> it might not be an actual problem in Airavata,
> >>>>>>> maybe some one more experienced with the codebase can point
this
> out.
> >>>>>>>
> >>>>>>
> >>>>>> All issues pointed out in the paper is not directly valid to
our
> >>>>>> conversion, I didn't list the issues actually need to address
in
> this
> >>>> case
> >>>>>> because thought it is worth to read that introduction part which
> >>>> explain
> >>>>>> the all the issues we have with this conversion and give us
a solid
> >>>>>> background of that.
> >>>>>>
> >>>>>>>  1. Anonymous values, Arrays, Implicit Typing, Character
sets -- I
> >>>>>> really
> >>>>>>>  dont see these as problems, as long as you can agree that
all
> >>>> parts of
> >>>>>>>  airavata will treat the JSON in a standard (probably we
have to
> >>>> define
> >>>>>>>  this) way.
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> The issue with JSON array only comes when we try to convert
XML to
> >>>> JSON not
> >>>>>> the other way. If we map with JSON, inputparameters and
> >>>> outputparameters in
> >>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore
we
> >>>> need to
> >>>>>> solve this issue.
> >>>>>>
> >>>>>> JSON XML JSON
> >>>>>> {"inputs":["test"]} --> <inputs>test<inputs>
 -->
> {"inputs":["test"]}
> >>>> //
> >>>>>> correct one
> >>>>>>                          --> {"inputs":"test"}     // incorrect
one
> >>>>>>
> >>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> >>>>>>
> >>>>>>>  Are separate namespaces used in Airavata? Only place I
can see
> >>>> this
> >>>>>>> being
> >>>>>>>  used is probably in the WSDL, but if we can agree on another
way
> >>>>>>>  of communicating registered applications' I/O parameters
to the
> >>>> front
> >>>>>>> end
> >>>>>>>  (JSON based), then maybe we can work around this (minor)
problem.
> >>>> Are
> >>>>>>>  custom processing instructions to the Xbaya XML parse even
used?
> >>>>>>>  3. Attributes -- Again, this can be fixed easily
> >>>>>>>
> >>>>>>
> >>>>>> Yes,attributes convertion will not be a big issues we can solve
it.
> As
> >>>>>> Lahiru mentioned in Hangout session namesapce handling is not
a big
> >>>> issue
> >>>>>> with Airavata.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> <array name="abc">
> >>>>>>>    <element>1</element>
> >>>>>>>    <element>2</element>
> >>>>>>>    <element>3</element>
> >>>>>>>    <element>4</element>
> >>>>>>> </array>
> >>>>>>>
> >>>>>>> Can become
> >>>>>>>
> >>>>>>> {
> >>>>>>>
> >>>>>>>   abc : ['1', '2', '3', '4']
> >>>>>>>
> >>>>>>> }
> >>>>>>>
> >>>>>>
> >>>>>> With this example it show us we need to change the XML message
> format
> >>>> of
> >>>>>> server side, which require to change the all schemas, If we
are
> going
> >>>> to
> >>>>>> change the schemas then we need to change the way it process
it in
> >>>> Ariavara
> >>>>>> core. We have dropped our initial major requirement, which is
keep
> the
> >>>>>> Airavata Server side as it is.
> >>>>>>
> >>>>>> with this conversion we only deal with json strings, yes we
can send
> >>>> JSON
> >>>>>> request with other formats supported by JSON like boolen, null,
> Number
> >>>>>> etc.. But there is no way to get the same JSON from XML as XML
only
> >>>> deal
> >>>>>> only with Strings. I think it is good if we can consume a this
> >> features
> >>>>>> with JSON.
> >>>>>>
> >>>>>> let say i need to send a integer or float to the server using
JSON
> >> then
> >>>>>> proper way is to send {"<name>":123.45} this will works
fine but
> >>>> problem is
> >>>>>> how we get the same output ?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Shameera.
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Subho.
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best Regards,
> >>>>>> Shameera Rathnayaka.
> >>>>>>
> >>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Best Regards,
> >>> Shameera Rathnayaka.
> >>>
> >>> Blog : http://shameerarathnayaka.blogspot.com/
> >>
> >>
> >
> >
> > --
> > Best Regards,
> > Shameera Rathnayaka.
> >
> > email: shameera AT apache.org , shameerainfo AT gmail.com
> > Blog : http://shameerarathnayaka.blogspot.com/
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message