airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Subho Banerjee <subs.z...@gmail.com>
Subject Re: Airavata GSoC 2013 Master Project
Date Thu, 02 May 2013 16:52:45 GMT
Hi Suresh,
I am in the process of writing the proposal... I should be done by tonight.
Will post it on the mailing list once I am done.

Cheers,
Subho


On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <smarru@apache.org> wrote:

> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
>
> Suresh
>
> On May 1, 2013, at 5:13 PM, Suresh Marru <smarru@apache.org> wrote:
>
> > Hi Vijayendra,
> >
> > As you can see from Shameera's proposal, he proposed a JSON conversion
> in front of WS Messenger. Also Danuska has been proposing for the AMQP and
> idea and deliberating its advantages. So given all these, I would suggest
> for you to keep focused on the UI aspects of the monitoring and write into
> your proposal a plan for determining a good strategy for the plumbing to
> WS-Eventing based existing system. You can have the concrete deliverables
> of new UI to change colors based on executions (as it currently does in
> XBaya), double click and show error messages and so forth. And keep it
> exploratory for the actually messaging format.
> >
> > I do not have any opinion on the libraries you mentioned, but yaa a ajax
> based pub system might be the right way to go. Pending the content format
> (JSON or WS-Eventing or JMS or AMQP or something else)
> >
> > Suresh
> >
> > On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> vijayendra.sdm@gmail.com> wrote:
> >
> >> Hi Suresh
> >>
> >> I am writing proposal for monitoring tool . The monitoring tool is
> based on
> >> pub-sub model (ws-messenger).
> >> While writing proposal , I have to back it by technical stuff that tells
> >> how can we achieve our purpose.
> >> As this monitoring tool is supposed to be a web based , and we are
> thinking
> >> in the lines of
> >> developing it in javascript.
> >> I was looking into javascript libraries that can we used with
> ws-messenger
> >> in the monitoring module.
> >> Please correct me if I am wrong.
> >> I came across some of the libraries
> >>
> >>  - jQuery custom
> >> events<
> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> >>  - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> >>  - PubSubJS <https://github.com/mroderick/PubSubJS>
> >>  - js-signals <http://millermedeiros.github.com/js-signals/>
> >>
> >> please tell me am I thinking in right direction?
> >>
> >> Regards
> >> Vijayendra
> >>
> >>
> >>
> >>
> >> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <smarru@apache.org> wrote:
> >>
> >>> Hi Shameera,
> >>>
> >>> This is great, I appreciate you sharing it, I realize this is still
> >>> working document, but I want other students to start seeing it and
> model
> >>> their proposals in a similar way.
> >>>
> >>> Airavata Mentors,
> >>>
> >>> Please provide feedback directly on the melange site and uncheck the
> >>> "private" box when you comment.
> >>>
> >>> Suresh
> >>>
> >>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> shameerainfo@gmail.com>
> >>> wrote:
> >>>
> >>>> Hi Suresh and All,
> >>>>
> >>>> Of course I am very much happy to share my proposal with everybody,
> >>>> actually i was going to update this thread with the melange link in
> few
> >>>> hours once i have done writing all the sections in the proposal. I
> >>> haven't
> >>>> yet added the milestone plan into it and now working on it.
> >>>>
> >>>> The sub area i am going to work from the Master project  is '
> >>> Implementing
> >>>> a JSON interface to Airavata Client side and Registry component'.
> Here is
> >>>> the link
> >>>>
> >>>
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> >>>> .
> >>>>
> >>>>
> >>>> Please note that i haven't completed everything in this and still
> doing
> >>>> modifications .Therefore proposal content may be changed bit, need to
> add
> >>>> more technical details of the approach which explains it well.
> >>>>
> >>>> I would like to know the feedback from all of you regarding the
> proposal
> >>>> and will be modifying it if there is anything to be done. Also please
> >>>> contact me if you need any help and i am very much willing to share
my
> >>>> thoughts with all.
> >>>>
> >>>> Thanks!
> >>>> Shameera
> >>>>
> >>>>
> >>>>
> >>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <smarru@apache.org>
> wrote:
> >>>>
> >>>>> Hi Shameera,
> >>>>>
> >>>>> Excellent proposal, great job. Would you mind to make  your proposal
> >>>>> public and post the link here? Your proposal should help others
to
> look
> >>> at
> >>>>> it and learn from.
> >>>>>
> >>>>> Again I emphasize to all students, please don't feel you will be
> >>> competing
> >>>>> with each others. If all of you write good proposals, there is a
good
> >>>>> chance all of you will be selected. But without a good proposal,
we
> >>> cannot
> >>>>> help.
> >>>>>
> >>>>> Suresh
> >>>>>
> >>>>>
> >>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> >>> shameerainfo@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> Yes it is not easy to solve all problems, But defining our own
> standard
> >>>>> or
> >>>>>> adhere to any standard
> >>>>>> provided by third party library will solve the problem to some
> extend.
> >>>>>>
> >>>>>> Here i see two possible approaches,
> >>>>>>
> >>>>>> 1. Use existing third party library(we can find which is best)
> adhere
> >>> to
> >>>>> it
> >>>>>> standard and see how we change the
> >>>>>> backend to be inline with it.
> >>>>>>
> >>>>>> 2. Use our own convention with help of XMLSchema (The way i
> suggest).
> >>>>>>
> >>>>>> As Suresh mentioned we can do a POC with both approaches to
compare
> >>>>>> performance
> >>>>>> and changes need to be done in server side. Then select the
best
> one.
> >>>>>>
> >>>>>> Another question was, can we works with graph data in JSON format.
> >>>>>> There are few JS graph framworks[1] which provide that
> functionality.
> >>>>>> we can use one of them to show airavata monitoring data as graphs
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Shameera.
> >>>>>>
> >>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/>
> ,
> >>>>>> Processing.js <http://processingjs.org> , Sencha
> >>>>>> Charts<http://www.sencha.com/products/extjs/>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <smarru@apache.org>
> >>> wrote:
> >>>>>>
> >>>>>>> Hi Vijeyandra,
> >>>>>>>
> >>>>>>> Airavata Messaging is based on a pub-sub model and the events
> >>> themselves
> >>>>>>> are xml (WS-Eventing [1]).
> >>>>>>>
> >>>>>>> The Messenger paper [2] should give you more information.
> >>>>>>>
> >>>>>>> Hi All (Especially those at WS02):
> >>>>>>>
> >>>>>>> Here is an old effort from a Morotuwa undergrad project,
you may
> want
> >>> to
> >>>>>>> read through these papers and chat with the authors to get
> >>> experiences:
> >>>>>>>
> >>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> >>>>>>>
> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> >>>>>>>
> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> >>>>>>> http://mooshabaya.wikidot.com/
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> >>>>>>>
> >>>>>>> Suresh
> >>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> >>>>>>> [2] -
> >>>>>>>
> >>>>>
> >>>
> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> >>>>>>>
> >>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> >>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> Hi Suresh
> >>>>>>>>
> >>>>>>>> I wanted to know more about the monitoring tool .
> >>>>>>>> Currently from where does the monitoring tool gets data
. Is it
> from
> >>>>>>>> workflow interpreter ? or Is it from the WS Messenger
( that might
> >>>>>>> continuously
> >>>>>>>> send messages to monitoring tool as to tell how much
is the
> progress
> >>>>>>>> and what are the variables getting changed) ?
> >>>>>>>>
> >>>>>>>> Again the how is the data being exchanged. I guess it
must be xml
> ?
> >>>>>>>> It must be one way data exchange . I mean the data is
TO the
> >>>>>>>> monitoring module.
> >>>>>>>> Then monitoring Tool  is sending back this
> >>>>>>>> data to Xbaya for displaying to the user ? Please correct
me if I
> am
> >>>>>>> wrong
> >>>>>>>>
> >>>>>>>> I have downloaded the source code from the trunk . can
you please
> >>> point
> >>>>>>>> me which part of code should I be code at for this module.
> >>>>>>>>
> >>>>>>>> Regards
> >>>>>>>> Vijayendra
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit
<
> >>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>> Hi
> >>>>>>>>
> >>>>>>>> What i am suggesting is, we send the JSON message directly
to
> >>> Airavata
> >>>>>>>> Backend(or Registry)
> >>>>>>>> When the message gets build after the transport phase,
convert
> JSON
> >>>>>>> message
> >>>>>>>> to SOAP(XML).
> >>>>>>>> From that point message will treated as SOAP message.
> >>>>>>>>
> >>>>>>>> If we look at the JSON <--> XML conversion there
are set of third
> >>> party
> >>>>>>>> libraries we
> >>>>>>>> can use for. But before selecting a one we need to think
about
> >>> problems
> >>>>>>>> having
> >>>>>>>>
> >>>>>>>> with JSON <--> XML and how these libraries handle
those issues.
> >>> Because
> >>>>>>> we
> >>>>>>>> need a robust
> >>>>>>>> way to do this conversions.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Shameera what you are suggesting is sending the JSON
message
> directly
> >>>>> to
> >>>>>>> Registry.
> >>>>>>>> when the message gets built after the transport phase
, convert
> it to
> >>>>>>> SOAP .
> >>>>>>>>
> >>>>>>>> If you are suggesting Registry will have JSON data instead
of
> WSDL ,
> >>>>>>> Then this might
> >>>>>>>> complicate the things  for us .
> >>>>>>>> The workflow interpreter needs wsdl(xml) to interpret
the
> workflows
> >>> and
> >>>>>>> for other details .
> >>>>>>>> Which means we might again have to do some changes with
workflow
> >>>>>>> interpretor .
> >>>>>>>> Yesterday from what I heard in discussion is that ,
they do not
> want
> >>> to
> >>>>>>> mess with workflow
> >>>>>>>> interpreter atleast for GSOC projects.
> >>>>>>>>
> >>>>>>>> What I want to suggest is , why carry the  JSON data
till
> Regisrty .
> >>>>>>> Build a interface
> >>>>>>>> before (Apache server API) where we  can do the necessary
> conversion
> >>>>>>> (JSON  to  SOAP).
> >>>>>>>> In this way we can avoid messing up with Airavata server
as a
> whole.
> >>>>>>>> Client ( using a we browser) is interacting with JSON
(web
> service)
> >>> but
> >>>>>>> the Apache server
> >>>>>>>> is interacting with SOAP.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Secondly yesterday Suresh was speaking about validating
the
> >>> connections
> >>>>>>> of the workflow.
> >>>>>>>> for example , the workflow is expecting a file as input
> >>>>>>>> but user is giving a sting  or int .
> >>>>>>>>
> >>>>>>>> Here what I suggest is , while creating wsdl in the
registry for a
> >>>>>>> particular
> >>>>>>>> workflow , we can add extra information in the form
of
> >>>>>>>> annotation as  the kind of input/ output the workflow
is
> accepting.
> >>>>>>>> Then we will be able to check these against users entry
during
> >>>>> execution.
> >>>>>>>> Please correct me if I am wrong.
> >>>>>>>>
> >>>>>>>> Regards
> >>>>>>>> Vijayendra
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
> subs.zero@gmail.com
> >>>>
> >>>>>>> wrote:
> >>>>>>>> Well exactly, as long as you can define standard way
of
> >>> communication.
> >>>>>>> That
> >>>>>>>> is, you can define in advance what should be a string,
array and
> what
> >>>>>>>> should be a integer etc. We have no problem.
> >>>>>>>>
> >>>>>>>> So, when you look at problems, with JSON <-> XML
or the other way
> >>>>> round,
> >>>>>>>> they talk of the very general case (where you no nothing
about the
> >>> data
> >>>>>>> you
> >>>>>>>> are converting other than it is valid XML/JSON). There
are a
> myriad
> >>> of
> >>>>>>>> problems in that case, which you pointed out.
> >>>>>>>>
> >>>>>>>> But when there is standard, there is only one way of
doing things,
> >>> and
> >>>>>>> not
> >>>>>>>> several. I think that is the way forward. So what I
am proposing
> is
> >>>>> maybe
> >>>>>>>> we all discuss and define this standard within the first
week of
> GSoC
> >>>>>>>> starting and then actually move into coding. So as long
as we work
> >>> with
> >>>>>>> the
> >>>>>>>> presumption that this will be done, we really dont have
to worry a
> >>> lot
> >>>>>>>> about this.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Subho.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka
<
> >>>>>>>> shameerainfo@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee
<
> >>> subs.zero@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Some of these problems are very specific to
what the XML is
> >>>>>>>>> representing,
> >>>>>>>>>> it might not be an actual problem in Airavata,
> >>>>>>>>>> maybe some one more experienced with the codebase
can point this
> >>> out.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> All issues pointed out in the paper is not directly
valid to our
> >>>>>>>>> conversion, I didn't list the issues actually need
to address in
> >>> this
> >>>>>>> case
> >>>>>>>>> because thought it is worth to read that introduction
part which
> >>>>>>> explain
> >>>>>>>>> the all the issues we have with this conversion
and give us a
> solid
> >>>>>>>>> background of that.
> >>>>>>>>>
> >>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing,
Character sets --
> I
> >>>>>>>>> really
> >>>>>>>>>> dont see these as problems, as long as you can
agree that all
> >>>>>>> parts of
> >>>>>>>>>> airavata will treat the JSON in a standard (probably
we have to
> >>>>>>> define
> >>>>>>>>>> this) way.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> The issue with JSON array only comes when we try
to convert XML
> to
> >>>>>>> JSON not
> >>>>>>>>> the other way. If we map with JSON, inputparameters
and
> >>>>>>> outputparameters in
> >>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays.
Therefore
> we
> >>>>>>> need to
> >>>>>>>>> solve this issue.
> >>>>>>>>>
> >>>>>>>>> JSON XML JSON
> >>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs>
 -->
> >>> {"inputs":["test"]}
> >>>>>>> //
> >>>>>>>>> correct one
> >>>>>>>>>                        --> {"inputs":"test"}
    // incorrect one
> >>>>>>>>>
> >>>>>>>>> 2. Namespaces, Processing Instructions -- Is this
required?
> >>>>>>>>>
> >>>>>>>>>> Are separate namespaces used in Airavata? Only
place I can see
> >>>>>>> this
> >>>>>>>>>> being
> >>>>>>>>>> used is probably in the WSDL, but if we can
agree on another way
> >>>>>>>>>> of communicating registered applications' I/O
parameters to the
> >>>>>>> front
> >>>>>>>>>> end
> >>>>>>>>>> (JSON based), then maybe we can work around
this (minor)
> problem.
> >>>>>>> Are
> >>>>>>>>>> custom processing instructions to the Xbaya
XML parse even used?
> >>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Yes,attributes convertion will not be a big issues
we can solve
> it.
> >>> As
> >>>>>>>>> Lahiru mentioned in Hangout session namesapce handling
is not a
> big
> >>>>>>> issue
> >>>>>>>>> with Airavata.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> <array name="abc">
> >>>>>>>>>>  <element>1</element>
> >>>>>>>>>>  <element>2</element>
> >>>>>>>>>>  <element>3</element>
> >>>>>>>>>>  <element>4</element>
> >>>>>>>>>> </array>
> >>>>>>>>>>
> >>>>>>>>>> Can become
> >>>>>>>>>>
> >>>>>>>>>> {
> >>>>>>>>>>
> >>>>>>>>>> abc : ['1', '2', '3', '4']
> >>>>>>>>>>
> >>>>>>>>>> }
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> With this example it show us we need to change the
XML message
> >>> format
> >>>>>>> of
> >>>>>>>>> server side, which require to change the all schemas,
If we are
> >>> going
> >>>>>>> to
> >>>>>>>>> change the schemas then we need to change the way
it process it
> in
> >>>>>>> Ariavara
> >>>>>>>>> core. We have dropped our initial major requirement,
which is
> keep
> >>> the
> >>>>>>>>> Airavata Server side as it is.
> >>>>>>>>>
> >>>>>>>>> with this conversion we only deal with json strings,
yes we can
> send
> >>>>>>> JSON
> >>>>>>>>> request with other formats supported by JSON like
boolen, null,
> >>> Number
> >>>>>>>>> etc.. But there is no way to get the same JSON from
XML as XML
> only
> >>>>>>> deal
> >>>>>>>>> only with Strings. I think it is good if we can
consume a this
> >>>>> features
> >>>>>>>>> with JSON.
> >>>>>>>>>
> >>>>>>>>> let say i need to send a integer or float to the
server using
> JSON
> >>>>> then
> >>>>>>>>> proper way is to send {"<name>":123.45} this
will works fine but
> >>>>>>> problem is
> >>>>>>>>> how we get the same output ?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Shameera.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Cheers,
> >>>>>>>>>> Subho.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Best Regards,
> >>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>
> >>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best Regards,
> >>>>>> Shameera Rathnayaka.
> >>>>>>
> >>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best Regards,
> >>>> Shameera Rathnayaka.
> >>>>
> >>>> email: shameera AT apache.org , shameerainfo AT gmail.com
> >>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>
> >>>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message