airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Marru <sma...@apache.org>
Subject Re: Airavata GSoC 2013 Master Project
Date Wed, 01 May 2013 12:00:42 GMT
Hi Shameera,

This is great, I appreciate you sharing it, I realize this is still working document, but
I want other students to start seeing it and model their proposals in a similar way. 

Airavata Mentors,

Please provide feedback directly on the melange site and uncheck the "private" box when you
comment.

Suresh

On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <shameerainfo@gmail.com> wrote:

> Hi Suresh and All,
> 
> Of course I am very much happy to share my proposal with everybody,
> actually i was going to update this thread with the melange link in few
> hours once i have done writing all the sections in the proposal. I haven't
> yet added the milestone plan into it and now working on it.
> 
> The sub area i am going to work from the Master project  is ' Implementing
> a JSON interface to Airavata Client side and Registry component'. Here is
> the link
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> .
> 
> 
> Please note that i haven't completed everything in this and still doing
> modifications .Therefore proposal content may be changed bit, need to add
> more technical details of the approach which explains it well.
> 
> I would like to know the feedback from all of you regarding the proposal
> and will be modifying it if there is anything to be done. Also please
> contact me if you need any help and i am very much willing to share my
> thoughts with all.
> 
> Thanks!
> Shameera
> 
> 
> 
> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <smarru@apache.org> wrote:
> 
>> Hi Shameera,
>> 
>> Excellent proposal, great job. Would you mind to make  your proposal
>> public and post the link here? Your proposal should help others to look at
>> it and learn from.
>> 
>> Again I emphasize to all students, please don't feel you will be competing
>> with each others. If all of you write good proposals, there is a good
>> chance all of you will be selected. But without a good proposal, we cannot
>> help.
>> 
>> Suresh
>> 
>> 
>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <shameerainfo@gmail.com>
>> wrote:
>> 
>>> Hi,
>>> 
>>> Yes it is not easy to solve all problems, But defining our own standard
>> or
>>> adhere to any standard
>>> provided by third party library will solve the problem to some extend.
>>> 
>>> Here i see two possible approaches,
>>> 
>>> 1. Use existing third party library(we can find which is best) adhere to
>> it
>>> standard and see how we change the
>>>   backend to be inline with it.
>>> 
>>> 2. Use our own convention with help of XMLSchema (The way i suggest).
>>> 
>>> As Suresh mentioned we can do a POC with both approaches to compare
>>> performance
>>> and changes need to be done in server side. Then select the best one.
>>> 
>>> Another question was, can we works with graph data in JSON format.
>>> There are few JS graph framworks[1] which provide that functionality.
>>> we can use one of them to show airavata monitoring data as graphs
>>> 
>>> Thanks,
>>> Shameera.
>>> 
>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/>
,
>>> Processing.js <http://processingjs.org> , Sencha
>>> Charts<http://www.sencha.com/products/extjs/>
>>> 
>>> 
>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <smarru@apache.org> wrote:
>>> 
>>>> Hi Vijeyandra,
>>>> 
>>>> Airavata Messaging is based on a pub-sub model and the events themselves
>>>> are xml (WS-Eventing [1]).
>>>> 
>>>> The Messenger paper [2] should give you more information.
>>>> 
>>>> Hi All (Especially those at WS02):
>>>> 
>>>> Here is an old effort from a Morotuwa undergrad project, you may want to
>>>> read through these papers and chat with the authors to get experiences:
>>>> 
>>>> http://dl.acm.org/citation.cfm?id=1890807
>>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
>>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
>>>> http://mooshabaya.wikidot.com/
>>>> 
>>>> 
>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
>>>> 
>>>> Suresh
>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
>>>> [2] -
>>>> 
>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
>>>> 
>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
>>>> vijayendra.sdm@gmail.com> wrote:
>>>> 
>>>>> Hi Suresh
>>>>> 
>>>>> I wanted to know more about the monitoring tool .
>>>>> Currently from where does the monitoring tool gets data . Is it from
>>>>> workflow interpreter ? or Is it from the WS Messenger ( that might
>>>> continuously
>>>>> send messages to monitoring tool as to tell how much is the progress
>>>>> and what are the variables getting changed) ?
>>>>> 
>>>>> Again the how is the data being exchanged. I guess it must be xml ?
>>>>> It must be one way data exchange . I mean the data is TO the
>>>>> monitoring module.
>>>>> Then monitoring Tool  is sending back this
>>>>> data to Xbaya for displaying to the user ? Please correct me if I am
>>>> wrong
>>>>> 
>>>>> I have downloaded the source code from the trunk . can you please point
>>>>> me which part of code should I be code at for this module.
>>>>> 
>>>>> Regards
>>>>> Vijayendra
>>>>> 
>>>>> 
>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
>>>> vijayendra.sdm@gmail.com> wrote:
>>>>> Hi
>>>>> 
>>>>> What i am suggesting is, we send the JSON message directly to Airavata
>>>>> Backend(or Registry)
>>>>> When the message gets build after the transport phase, convert JSON
>>>> message
>>>>> to SOAP(XML).
>>>>> From that point message will treated as SOAP message.
>>>>> 
>>>>> If we look at the JSON <--> XML conversion there are set of third
party
>>>>> libraries we
>>>>> can use for. But before selecting a one we need to think about problems
>>>>> having
>>>>> 
>>>>> with JSON <--> XML and how these libraries handle those issues.
Because
>>>> we
>>>>> need a robust
>>>>> way to do this conversions.
>>>>> 
>>>>> 
>>>>> 
>>>>> Shameera what you are suggesting is sending the JSON message directly
>> to
>>>> Registry.
>>>>> when the message gets built after the transport phase , convert it to
>>>> SOAP .
>>>>> 
>>>>> If you are suggesting Registry will have JSON data instead of WSDL ,
>>>> Then this might
>>>>> complicate the things  for us .
>>>>> The workflow interpreter needs wsdl(xml) to interpret the workflows and
>>>> for other details .
>>>>> Which means we might again have to do some changes with workflow
>>>> interpretor .
>>>>> Yesterday from what I heard in discussion is that , they do not want
to
>>>> mess with workflow
>>>>> interpreter atleast for GSOC projects.
>>>>> 
>>>>> What I want to suggest is , why carry the  JSON data till Regisrty .
>>>> Build a interface
>>>>> before (Apache server API) where we  can do the necessary conversion
>>>> (JSON  to  SOAP).
>>>>> In this way we can avoid messing up with Airavata server as a whole.
>>>>> Client ( using a we browser) is interacting with JSON (web service) but
>>>> the Apache server
>>>>> is interacting with SOAP.
>>>>> 
>>>>> 
>>>>> 
>>>>> Secondly yesterday Suresh was speaking about validating the connections
>>>> of the workflow.
>>>>> for example , the workflow is expecting a file as input
>>>>> but user is giving a sting  or int .
>>>>> 
>>>>> Here what I suggest is , while creating wsdl in the registry for a
>>>> particular
>>>>> workflow , we can add extra information in the form of
>>>>> annotation as  the kind of input/ output the workflow is accepting.
>>>>> Then we will be able to check these against users entry during
>> execution.
>>>>> Please correct me if I am wrong.
>>>>> 
>>>>> Regards
>>>>> Vijayendra
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <subs.zero@gmail.com>
>>>> wrote:
>>>>> Well exactly, as long as you can define standard way of communication.
>>>> That
>>>>> is, you can define in advance what should be a string, array and what
>>>>> should be a integer etc. We have no problem.
>>>>> 
>>>>> So, when you look at problems, with JSON <-> XML or the other way
>> round,
>>>>> they talk of the very general case (where you no nothing about the data
>>>> you
>>>>> are converting other than it is valid XML/JSON). There are a myriad of
>>>>> problems in that case, which you pointed out.
>>>>> 
>>>>> But when there is standard, there is only one way of doing things, and
>>>> not
>>>>> several. I think that is the way forward. So what I am proposing is
>> maybe
>>>>> we all discuss and define this standard within the first week of GSoC
>>>>> starting and then actually move into coding. So as long as we work with
>>>> the
>>>>> presumption that this will be done, we really dont have to worry a lot
>>>>> about this.
>>>>> 
>>>>> Cheers,
>>>>> Subho.
>>>>> 
>>>>> 
>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>>>>> shameerainfo@gmail.com> wrote:
>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <subs.zero@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Some of these problems are very specific to what the XML is
>>>>>> representing,
>>>>>>> it might not be an actual problem in Airavata,
>>>>>>> maybe some one more experienced with the codebase can point this
out.
>>>>>>> 
>>>>>> 
>>>>>> All issues pointed out in the paper is not directly valid to our
>>>>>> conversion, I didn't list the issues actually need to address in
this
>>>> case
>>>>>> because thought it is worth to read that introduction part which
>>>> explain
>>>>>> the all the issues we have with this conversion and give us a solid
>>>>>> background of that.
>>>>>> 
>>>>>>>  1. Anonymous values, Arrays, Implicit Typing, Character sets
-- I
>>>>>> really
>>>>>>>  dont see these as problems, as long as you can agree that all
>>>> parts of
>>>>>>>  airavata will treat the JSON in a standard (probably we have
to
>>>> define
>>>>>>>  this) way.
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> The issue with JSON array only comes when we try to convert XML to
>>>> JSON not
>>>>>> the other way. If we map with JSON, inputparameters and
>>>> outputparameters in
>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore we
>>>> need to
>>>>>> solve this issue.
>>>>>> 
>>>>>> JSON XML JSON
>>>>>> {"inputs":["test"]} --> <inputs>test<inputs>  -->
{"inputs":["test"]}
>>>> //
>>>>>> correct one
>>>>>>                          --> {"inputs":"test"}     // incorrect
one
>>>>>> 
>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
>>>>>> 
>>>>>>>  Are separate namespaces used in Airavata? Only place I can see
>>>> this
>>>>>>> being
>>>>>>>  used is probably in the WSDL, but if we can agree on another
way
>>>>>>>  of communicating registered applications' I/O parameters to
the
>>>> front
>>>>>>> end
>>>>>>>  (JSON based), then maybe we can work around this (minor) problem.
>>>> Are
>>>>>>>  custom processing instructions to the Xbaya XML parse even used?
>>>>>>>  3. Attributes -- Again, this can be fixed easily
>>>>>>> 
>>>>>> 
>>>>>> Yes,attributes convertion will not be a big issues we can solve it.
As
>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a big
>>>> issue
>>>>>> with Airavata.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> <array name="abc">
>>>>>>>    <element>1</element>
>>>>>>>    <element>2</element>
>>>>>>>    <element>3</element>
>>>>>>>    <element>4</element>
>>>>>>> </array>
>>>>>>> 
>>>>>>> Can become
>>>>>>> 
>>>>>>> {
>>>>>>> 
>>>>>>>   abc : ['1', '2', '3', '4']
>>>>>>> 
>>>>>>> }
>>>>>>> 
>>>>>> 
>>>>>> With this example it show us we need to change the XML message format
>>>> of
>>>>>> server side, which require to change the all schemas, If we are going
>>>> to
>>>>>> change the schemas then we need to change the way it process it in
>>>> Ariavara
>>>>>> core. We have dropped our initial major requirement, which is keep
the
>>>>>> Airavata Server side as it is.
>>>>>> 
>>>>>> with this conversion we only deal with json strings, yes we can send
>>>> JSON
>>>>>> request with other formats supported by JSON like boolen, null, Number
>>>>>> etc.. But there is no way to get the same JSON from XML as XML only
>>>> deal
>>>>>> only with Strings. I think it is good if we can consume a this
>> features
>>>>>> with JSON.
>>>>>> 
>>>>>> let say i need to send a integer or float to the server using JSON
>> then
>>>>>> proper way is to send {"<name>":123.45} this will works fine
but
>>>> problem is
>>>>>> how we get the same output ?
>>>>>> 
>>>>>> Thanks,
>>>>>> Shameera.
>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Subho.
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Best Regards,
>>>>>> Shameera Rathnayaka.
>>>>>> 
>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> --
>>> Best Regards,
>>> Shameera Rathnayaka.
>>> 
>>> Blog : http://shameerarathnayaka.blogspot.com/
>> 
>> 
> 
> 
> -- 
> Best Regards,
> Shameera Rathnayaka.
> 
> email: shameera AT apache.org , shameerainfo AT gmail.com
> Blog : http://shameerarathnayaka.blogspot.com/


Mime
View raw message