Return-Path: X-Original-To: apmail-airavata-dev-archive@www.apache.org Delivered-To: apmail-airavata-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7BC2AF65A for ; Thu, 25 Apr 2013 20:39:30 +0000 (UTC) Received: (qmail 36788 invoked by uid 500); 25 Apr 2013 20:39:30 -0000 Delivered-To: apmail-airavata-dev-archive@airavata.apache.org Received: (qmail 36748 invoked by uid 500); 25 Apr 2013 20:39:30 -0000 Mailing-List: contact dev-help@airavata.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airavata.apache.org Delivered-To: mailing list dev@airavata.apache.org Received: (qmail 36740 invoked by uid 99); 25 Apr 2013 20:39:30 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Apr 2013 20:39:30 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of vijayendra.sdm@gmail.com designates 209.85.215.49 as permitted sender) Received: from [209.85.215.49] (HELO mail-la0-f49.google.com) (209.85.215.49) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Apr 2013 20:39:23 +0000 Received: by mail-la0-f49.google.com with SMTP id fp13so728718lab.8 for ; Thu, 25 Apr 2013 13:39:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=ULq/hR6gNoP9+VnjGIQaYiya1T26EFMyMflLmZ49LM4=; b=xEQeI5NO/6zOVO/EZ+jDcI64iQNthzpruABMvMbgjrs5i3r5uEexaPyfihEDdWZytM WEbn7Vi3zxk9AGxEyu4RuQFspQgv8kPuocj87NE1RuTq/HdazdVMvjcmiG60SbWfFo4A nwDTmstWPf0OgsUMgASgOkaLSY+pyLrmDw/W7ZDLAnqalrOt+LlmYQoKfeq7UdRlvjDb BZ8nE49f7sg2Wbgm2yRXcjlXyPmzKTUJHLqlvDsw0KI9dbzkQsxWrATKhX/gsOLfbUwn sVBs9GiL8FQelsjgz3sChTUVTjhBaLynrrLaiMtP5hCN2gk+yo2MJBD7f6rIrLsf58XI p6yw== MIME-Version: 1.0 X-Received: by 10.152.4.131 with SMTP id k3mr21092470lak.26.1366922340883; Thu, 25 Apr 2013 13:39:00 -0700 (PDT) Received: by 10.112.19.66 with HTTP; Thu, 25 Apr 2013 13:39:00 -0700 (PDT) In-Reply-To: References: <516DB16D.6070508@iu.edu> <3744C349-EBC6-4508-A60F-F4E9A818DDBC@apache.org> <14CB1E43-F5B4-4FEF-A8A0-214277F4889D@apache.org> Date: Fri, 26 Apr 2013 02:09:00 +0530 Message-ID: Subject: Re: Airavata GSoC 2013 Master Project From: Vijayendra Grampurohit To: dev@airavata.apache.org Content-Type: multipart/related; boundary=089e013d173090609304db35681d X-Virus-Checked: Checked by ClamAV on apache.org --089e013d173090609304db35681d Content-Type: multipart/alternative; boundary=089e013d173090609104db35681c --089e013d173090609104db35681c Content-Type: text/plain; charset=ISO-8859-1 Hi 1] Currently the workflow description has the graph representation form xbaya . The data for the Workflow Inputs come from the Registry .This data is in wsdl form which is stored in the Registry. There is also some metadata associated with wsdl . So that the client which has to bind the input values can use it. As we are discussing in the line of developing a browser based application. For constructing workflow we can make use of below graph library's http://jsplumbtoolkit.com/jquery/demo.html http://raphaeljs.com/ In our project( Openshift workflows) we had used jsplumb library which worked well with Angularjs . There are many more graph libraries which also can be explored. 2] Currently the data that is stored in Registry is of the form wsdl. The workflow will use the data stored in the Registry . For the new workflow as there is a discussion going on what kind of data are we going to store in registry (i.e wsdl or JSON ). This can be worked accordingly. 3] Monitoring tool : I am thinking in terms of a browser plugin or a simple java script based web based monitoring which will notify users on workflow progress in real time. This can be developed as a separate module . The Monitoring tool subscribes to a pre-specified topic to which Workflow Engine and GFac are publishing status notifications.Then the monitoring tool translates these messages and shows to to the user in the front end. Please see the image inserted. [image: Inline image 1] What we can also do , When user is executing a very large workflow with hundreds of variables and doesn't want to track every thing, Then we can have a option to which allows customization in the monitoring tool . My point is we can have features in which user can monitor the variables or data he wants. Please correct me if I am wrong any where. Regards Vijayendra On Thu, Apr 25, 2013 at 7:08 PM, Suresh Marru wrote: > On Apr 23, 2013, at 8:01 AM, Suresh Marru wrote: > > > Great discussion Shameera & Subho. > > > > Looks like you guys have a fair idea on what needs to be done, as you > both state, a week or two into GSOC, you can narrow down on design and > specifications and so forth. Given there are multiple students tackling > these issues collectively, if there are hard decisions to make, I would > suggest to try two approaches in parallel and quickly pick one after a > proof of concept. > > Can you all discuss the workflow composition strategies related to the > data model the same way we discussed application descriptions? > > We need to decompose the master project into smaller, well aligned > projects this weekend. So please start posting ideas on how to sub-divide > the projects. > > Suresh > > > > > Suresh > > > > On Apr 23, 2013, at 3:43 AM, Subho Banerjee wrote: > > > >> Well exactly, as long as you can define standard way of communication. > That > >> is, you can define in advance what should be a string, array and what > >> should be a integer etc. We have no problem. > >> > >> So, when you look at problems, with JSON <-> XML or the other way round, > >> they talk of the very general case (where you no nothing about the data > you > >> are converting other than it is valid XML/JSON). There are a myriad of > >> problems in that case, which you pointed out. > >> > >> But when there is standard, there is only one way of doing things, and > not > >> several. I think that is the way forward. So what I am proposing is > maybe > >> we all discuss and define this standard within the first week of GSoC > >> starting and then actually move into coding. So as long as we work with > the > >> presumption that this will be done, we really dont have to worry a lot > >> about this. > >> > >> Cheers, > >> Subho. > >> > >> > >> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka < > >> shameerainfo@gmail.com> wrote: > >> > >>> Hi, > >>> > >>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee > >>> wrote: > >>> > >>>> Some of these problems are very specific to what the XML is > >>> representing, > >>>> it might not be an actual problem in Airavata, > >>>> maybe some one more experienced with the codebase can point this out. > >>>> > >>> > >>> All issues pointed out in the paper is not directly valid to our > >>> conversion, I didn't list the issues actually need to address in this > case > >>> because thought it is worth to read that introduction part which > explain > >>> the all the issues we have with this conversion and give us a solid > >>> background of that. > >>> > >>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets -- I > >>> really > >>>> dont see these as problems, as long as you can agree that all parts > of > >>>> airavata will treat the JSON in a standard (probably we have to > define > >>>> this) way. > >>>> > >>> > >>> > >>> The issue with JSON array only comes when we try to convert XML to > JSON not > >>> the other way. If we map with JSON, inputparameters and > outputparameters in > >>> the ServiceDescription.xsd will map with JSON Arrays. Therefore we > need to > >>> solve this issue. > >>> > >>> JSON XML JSON > >>> {"inputs":["test"]} --> test --> {"inputs":["test"]} > // > >>> correct one > >>> --> {"inputs":"test"} // incorrect one > >>> > >>> 2. Namespaces, Processing Instructions -- Is this required? > >>> > >>>> Are separate namespaces used in Airavata? Only place I can see this > >>>> being > >>>> used is probably in the WSDL, but if we can agree on another way > >>>> of communicating registered applications' I/O parameters to the front > >>>> end > >>>> (JSON based), then maybe we can work around this (minor) problem. Are > >>>> custom processing instructions to the Xbaya XML parse even used? > >>>> 3. Attributes -- Again, this can be fixed easily > >>>> > >>> > >>> Yes,attributes convertion will not be a big issues we can solve it. As > >>> Lahiru mentioned in Hangout session namesapce handling is not a big > issue > >>> with Airavata. > >>> > >>> > >>> > >>>> > >>>> > >>>> 1 > >>>> 2 > >>>> 3 > >>>> 4 > >>>> > >>>> > >>>> Can become > >>>> > >>>> { > >>>> > >>>> abc : ['1', '2', '3', '4'] > >>>> > >>>> } > >>>> > >>> > >>> With this example it show us we need to change the XML message format > of > >>> server side, which require to change the all schemas, If we are going > to > >>> change the schemas then we need to change the way it process it in > Ariavara > >>> core. We have dropped our initial major requirement, which is keep the > >>> Airavata Server side as it is. > >>> > >>> with this conversion we only deal with json strings, yes we can send > JSON > >>> request with other formats supported by JSON like boolen, null, Number > >>> etc.. But there is no way to get the same JSON from XML as XML only > deal > >>> only with Strings. I think it is good if we can consume a this features > >>> with JSON. > >>> > >>> let say i need to send a integer or float to the server using JSON then > >>> proper way is to send {"":123.45} this will works fine but > problem is > >>> how we get the same output ? > >>> > >>> Thanks, > >>> Shameera. > >>> > >>> > >>>> > >>>> > >>>> Cheers, > >>>> Subho. > >>>> > >>> > >>> > >>> > >>> -- > >>> Best Regards, > >>> Shameera Rathnayaka. > >>> > >>> Blog : http://shameerarathnayaka.blogspot.com/ > >>> > > > > --089e013d173090609104db35681c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi

1] Currently t= he workflow description has the graph representation form =A0xbaya .
<= div>The data for the Workflow Inputs come from the Registry .This data is i= n wsdl form which is stored=A0
in the Registry.
There is also some metadata associated with= wsdl . So that the client which has to bind the input values can use it.

As we are discussing in the line of developing a = =A0browser based application.
For constructing workflow we can make use of below graph library's= =A0
In our project( Openshift workflows) we had used jsplumb library=A0
which worked well with Angularjs .

There a= re many more graph libraries which also can be explored.

2] Currently the data that is stored in Registry is of the form wsdl.= =A0
The workflow will use the data stored in the Registry .
=
For the new workflow as there is a discussion going on what kind of da= ta are we=A0
going to store in registry =A0 (i.e wsdl or JSON ). This can be worked= =A0
accordingly.

3] Monitoring tool : I = am thinking in terms of a=A0
browser plugin or a simple java scri= pt based web based monitoring which will=A0
notify users on workflow progress in real time.
This c= an be developed as a separate module .=A0
The Monitoring tool subscribes to a pre-specified top= ic to which=A0
Workflow Engine and GFac are pu= blishing status notifications.Then the monitoring
tool translates these messages and s= hows to to the user in the front end.
Please see the image inserted.

3D"Inline

What we can also do , When use= r is executing a very large workflow
with hundreds of varia= bles and doesn't want to track every thing, Then =A0we can have a optio= n to=A0
which allows customization in the monitoring tool .
My point is =A0we can have =A0features in which user can monitor=A0
the variables or data he wants.

Please correct me if I am wrong any where.

Regards
Vijayendra

=A0



On Thu, Apr 25, 2013 at 7:08 PM, Suresh Marru <smarru@apache.org> wrote:
On Apr 23, 2013, at 8:01 AM, Suresh Marru <smarru@apache.org> wrote:

> Great discussion Shameera & Subho.
>
> Looks like you guys have a fair idea on what needs to be done, as you = both state, a week or two into GSOC, you can narrow down on design and spec= ifications and so forth. Given there are multiple students tackling these i= ssues collectively, if there are hard decisions to make, I would suggest to= try two approaches in parallel and quickly pick one after a proof of conce= pt.

Can you all discuss the workflow composition strategies related to th= e data model the same way we discussed application descriptions?

We need to decompose the master project into smaller, well aligned projects= this weekend. So please start posting ideas on how to sub-divide the proje= cts.

Suresh

>
> Suresh
>
> On Apr 23, 2013, at 3:43 AM, Subho Banerjee <subs.zero@gmail.com> wrote:
>
>> Well exactly, as long as you can define standard way of communicat= ion. That
>> is, you can define in advance what should be a string, array and w= hat
>> should be a integer etc. We have no problem.
>>
>> So, when you look at problems, with JSON <-> XML or the othe= r way round,
>> they talk of the very general case (where you no nothing about the= data you
>> are converting other than it is valid XML/JSON). There are a myria= d of
>> problems in that case, which you pointed out.
>>
>> But when there is standard, there is only one way of doing things,= and not
>> several. I think that is the way forward. So what I am proposing i= s maybe
>> we all discuss and define this standard within the first week of G= SoC
>> starting and then actually move into coding. So as long as we work= with the
>> presumption that this will be done, we really dont have to worry a= lot
>> about this.
>>
>> Cheers,
>> Subho.
>>
>>
>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>> shameerainfo@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <subs.zero@gmail.com>
>>> wrote:
>>>
>>>> Some of these problems are very specific to what the XML i= s
>>> representing,
>>>> it might not be an actual problem in Airavata,
>>>> maybe some one more experienced with the codebase can poin= t this out.
>>>>
>>>
>>> All issues pointed out in the paper is not directly valid to o= ur
>>> conversion, I didn't list the issues actually need to addr= ess in this case
>>> because thought it is worth to read that introduction part whi= ch explain
>>> the all the issues we have with this conversion and give us a = solid
>>> background of that.
>>>
>>>> =A01. Anonymous values, Arrays, Implicit Typing, Character= sets -- I
>>> really
>>>> =A0dont see these as problems, as long as you can agree th= at all parts of
>>>> =A0airavata will treat the JSON in a standard (probably we= have to define
>>>> =A0this) way.
>>>>
>>>
>>>
>>> The issue with JSON array only comes when we try to convert XM= L to JSON not
>>> the other way. If we map with JSON, inputparameters and output= parameters in
>>> the ServiceDescription.xsd will map with JSON Arrays. Therefor= e we need to
>>> solve this issue.
>>>
>>> JSON XML JSON
>>> {"inputs":["test"]} --> <inputs>t= est<inputs> =A0--> {"inputs":["test"]} =A0 //<= br> >>> correct one
>>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0--> {&qu= ot;inputs":"test"} =A0 =A0 // incorrect one
>>>
>>> 2. Namespaces, Processing Instructions -- Is this required? >>>
>>>> =A0Are separate namespaces used in Airavata? Only place I = can see this
>>>> being
>>>> =A0used is probably in the WSDL, but if we can agree on an= other way
>>>> =A0of communicating registered applications' I/O param= eters to the front
>>>> end
>>>> =A0(JSON based), then maybe we can work around this (minor= ) problem. Are
>>>> =A0custom processing instructions to the Xbaya XML parse e= ven used?
>>>> =A03. Attributes -- Again, this can be fixed easily
>>>>
>>>
>>> Yes,attributes convertion will not be a big issues we can solv= e it. As
>>> Lahiru mentioned in Hangout session namesapce handling is not = a big issue
>>> with Airavata.
>>>
>>>
>>>
>>>>
>>>> <array name=3D"abc">
>>>> =A0 =A0<element>1</element>
>>>> =A0 =A0<element>2</element>
>>>> =A0 =A0<element>3</element>
>>>> =A0 =A0<element>4</element>
>>>> </array>
>>>>
>>>> Can become
>>>>
>>>> {
>>>>
>>>> =A0 abc : ['1', '2', '3', '4&#= 39;]
>>>>
>>>> }
>>>>
>>>
>>> With this example it show us we need to change the XML message= format of
>>> server side, which require to change the all schemas, If we ar= e going to
>>> change the schemas then we need to change the way it process i= t in Ariavara
>>> core. We have dropped our initial major requirement, which is = keep the
>>> Airavata Server side as it is.
>>>
>>> with this conversion we only deal with json strings, yes we ca= n send JSON
>>> request with other formats supported by JSON like boolen, null= , Number
>>> etc.. But there is no way to get the same JSON from XML as XML= only deal
>>> only with Strings. I think it is good if we can consume a this= features
>>> with JSON.
>>>
>>> let say i need to send a integer or float to the server using = JSON then
>>> proper way is to send {"<name>":123.45} this w= ill works fine but problem is
>>> how we get the same output ?
>>>
>>> Thanks,
>>> Shameera.
>>>
>>>
>>>>
>>>>
>>>> Cheers,
>>>> Subho.
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Shameera Rathnayaka.
>>>
>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>
>


--089e013d173090609104db35681c-- --089e013d173090609304db35681d--