incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chathura Herath <chathu...@gmail.com>
Subject Re: [PROPOSAL] Airavata for the incubator
Date Fri, 29 Apr 2011 01:10:04 GMT
Hi,

I think this would be a good move going forward.In addition to points
Glen and Srinath raised also Airavata has a significant stake in So
going forWS-Messenger in terms of monitoring of long workflows across
resource boundaries as well as the components being orchestrated.
Going forward it will become hardened module for axis2. Also
WS-Messsenger support WS-brokered notification.

I am +1 to the idea of bringing WS-messenger into Airavata and moving
it to Savan before graduation.



Thanks
Chathura


On Thu, Apr 28, 2011 at 8:21 PM, Srinath Perera <srinath@wso2.com> wrote:
> Hi Glen,
>
> There is a slight difference between Savan and messenger as one let
> you subscribe to event generated by services whereas other is a
> broker. Still I agree that messenger is best placed inside Savan.
>
> I would like to propose that we bring in ws-messenger into Airavata
> and move it off before the graduation.
>
> --Srinath
>
>
> On Fri, Apr 29, 2011 at 3:51 AM, Glen Daniels <glen@thoughtcraft.com> wrote:
>> Hi Ross, all,
>>
>> I think in general this looks great and I'd definitely +1 bringing Airavata
>> to the Incubator.
>>
>> The only point I'd explore a bit is the WS-Messenger component, in
>> particular whether this bit might (either now or eventually) want to end up
>> in the Axis project, since it seems like a worthy replacement to the
>> not-very-well-maintained Savan (the WS-Eventing-over-Axis2 module in Axis).
>>
>> While it's certainly not necessary to have all Axis2 modules in the same
>> place - there are plenty at WSO2 and elsewhere - it is pretty nice for both
>> users and developers to have some of the core ones there.  For users, it
>> makes them easy to find, and for developers, it makes it easier to test
>> everything together and ensure that Axis2 changes don't break any
>> functionality in the extension modules.
>>
>> Since a bunch of the Airavata team are already Axis committers, I believe it
>> should be possible to simply check WS-Messenger in there if the team (and
>> the Axis team) agrees that would be a good home. Alternately Airavata could
>> go through the incubator as-is and make a decision about this during
>> graduation (at the risk of package-naming challenges).
>>
>> Cool stuff - I remember Eran talking about this a couple of years back.
>>
>> Thanks,
>> --Glen
>>
>> On 4/23/11 5:53 PM, Ross Gardler wrote:
>>>
>>> I would like to propose Airavata for entry into the Apache Incubator.
>>>
>>> The full proposal can be found at [1] and is copied at the end of this
>>> mail. For those in a hurry here's a quick summary:
>>>
>>> Airavata is a software toolkit currently used to build science gateways
>>> but that has a much wider potential use. It provides features to
>>> compose, manage, execute, and monitor large scale applications and
>>> workflows on computational resources ranging from local clusters to
>>> national grids and computing clouds. Users can use Airavata back end
>>> services and build gadgets to deploy in open social containers such as
>>> Apache Rave and modify them to suite their needs. Airavata builds on
>>> general concepts of service oriented computing, distributed messaging,
>>> and workflow composition and orchestration.
>>>
>>> Airavata will provide web interfaces and scalable Service Oriented
>>> Architecture based backend services to build or enhance Science Gateway
>>> (see https://www.teragrid.org/web/science-gateways/) and similar
>>> environments. Airavata will specifically focus on:
>>>
>>> 1. sophisticated server-side tools for registering and managing large
>>> scale applications on computational resources.
>>>
>>> 2. graphical user interfaces to construct, execute, control, manage and
>>> reuse of scientific workflows.
>>>
>>> 3. interfacing and interoperability with with various external (third
>>> party) data and provenance management tools
>>>
>>> The project team consists of a number of existing Apache Committers and
>>> the code comes from the same stable as some of the code donated to
>>> Apache Rave (Incubating).
>>>
>>> We welcome your questions, suggestions, observations and support.
>>>
>>> Ross
>>>
>>> [1] http://wiki.apache.org/incubator/AiravataProposal
>>>
>>> FULL PROPOSAL TEXT
>>> ==================
>>>
>>> = Airavata Proposal for Apache Incubator =
>>>
>>> == Abstract ==
>>> Airavata is a software toolkit currently used to build science
>>> gateways but that has a much wider potential use. It provides features
>>> to compose, manage, execute, and monitor large scale applications and
>>> workflows on computational resources ranging from local clusters to
>>> national grids and computing clouds. Users can use Airavata back end
>>> services and build gadgets to deploy in open social containers such as
>>> Apache Rave and modify them to suite their needs. Airavata builds on
>>> general concepts of service oriented computing, distributed messaging,
>>> and workflow composition and orchestration.
>>>
>>>
>>> == Proposal ==
>>>
>>> Airavata will provide web interfaces and scalable Service Oriented
>>> Architecture based backend services to build or enhance Science
>>> Gateway (see https://www.teragrid.org/web/science-gateways/)
>>> and similar environments. Airavata will specifically focus on:
>>>
>>> 1. sophisticated server-side tools for registering and managing large
>>> scale applications on computational resources.
>>> 2. graphical user interfaces to construct, execute, control, manage and
>>> reuse of scientific workflows.
>>> 3. interfacing and interoperability with with various external (third
>>> party) data and provenance management tools.
>>>
>>> == Background ==
>>> Working in close quarters with Apache Axis2 committers and inspired by
>>> the true open source community driven software development of ASF,
>>> Suresh Marru and Marlon Pierce have been pioneering the idea of a
>>> Science Gateways software-based Apache project since late 2008. Many
>>> Apache members have fostered these ideas and guided them to arrive at
>>> this proposal.
>>>
>>> Currently the software is a actively used in various science
>>> gateways. But the tools are general purpose and build upon widely used
>>> Apache tools like Axis2, ODE engine. The core team is motivated to
>>> expand the community and build a community welcoming both synergistic
>>> software components and also new usage scenarios.
>>>
>>> It is perhaps worth noting that one of the three seed projects that
>>> make up the Apache Rave (Incubating) project is also the product of
>>> this same team and is derived from the same Science Gateways
>>> community.
>>>
>>> == Rationale ==
>>>
>>> The nature of computational problems has evolved from simple desktop
>>> calculations to complex, multidisciplinary activities that require the
>>> monitoring and analysis of remote data streams, database and web
>>> search and large ensembles of simulations. In the academic domain
>>> Science Gateways have emerged to address these needs and have built
>>> software platforms that provide a community of users with the ability
>>> to easily solve computational problems within a specific domain. The
>>> tools developed to support these gateways are potentially of value to
>>> any organisation needing to perform complex computations. Gateways
>>> provide a convenient interface to the underlying infrastrucure without
>>> the need for a deep understanding of the intricacies that
>>> infrastructure.
>>>
>>> We summarize the rationale for choosing The Apache Software Foundation
>>> (ASF) below. This is what we hope to gain from participating in the
>>> ASF.
>>>
>>> 1. '''Broader impact''': our science gateway tool set is based on
>>> Service Oriented Architecture principles, and it has always been our
>>> goal to align our software with broader trends in the development of
>>> software for distributed systems. Participating in the ASF provides a
>>> concrete way to implement this idea. In particular, we have done
>>> extensive work on the workflow systems, messaging, and application
>>> management as Web services from the perspective of computational science
>>> use cases (i.e., high failure rates, very long running jobs, dynamic
>>> service creation, workflows not expressible as directed acyclic graphs,
>>> etc). These requirements and our work to implement them have already had
>>> direct impact on the Apache Axis 2 and Apache ODE projects. As an Apache
>>> project, it is hoped that our community will have an enhanced
>>> opportunity for collaboration and complementary development with Apache
>>> Hadoop (for scientific application management), Apache QPID (for
>>> messaging), Apache Rave (incubator - Open Social Container) and others.
>>> It is our goal to expand our software’s usage beyond just science
>>> gateways to the broader enterprise community.
>>> 2. '''Sustainability''': Science gateway software development (and
>>> cyberinfrastructure software generally) is primarily funded in the US by
>>> the National Science Foundation (NSF), so the long term sustainability
>>> of software across funding cycles is a longstanding problem. The NSF is
>>> attempting to solve this problem, and its vision for sustainable
>>> software is described here:
>>> http://www.nsf.gov/pubs/2010/nsf10015/nsf10015.jsp. Participating in the
>>> ASF is our project’s vision for reaching software sustainability that
>>> underpins the NSF CF21 vision. As a successful ASF project (after
>>> incubation), we will have created a community led, rather than funding
>>> led, environment for the development of our sotware. This community,
>>> through our community engagement work and adoption of meritocratic
>>> principles, will expand beyond our current core team and existing
>>> project collaborations. This will greatly increase the chances that our
>>> software will continue to grow and improve beyond the participation of
>>> any individuals.
>>> 3. '''Maturity''': much of the software included in this proposal was
>>> developed initially by graduate students as part of their Ph. D. work.
>>> The Open Grid Computing Environment has devoted significant effort
>>> (through salaried staff and volunteers from collaborating institutions)
>>> to convert these research projects into mature, reliable, well-written,
>>> packaged components. The code is currently hosted at SourceForge, but we
>>> recognize the need to go beyond just the SourceForge support tools to
>>> participate in a real community of software engineering experts. It is
>>> our desire, through the Apache Incubator, to take our software
>>> engineering efforts to a higher level by learning from the substantial
>>> experience of appropraite Apache Committers. Apache mentors will provide
>>> initial guidance, as will the attraction of additional committers from
>>> the relevant Apache projects.
>>>
>>> == Initial Goals ==
>>>
>>> * Implement a standalone version of the code base with a simple hello
>>> world service, workflow and gadget(s) to access the examples.
>>> * Migration of documentation and design knowledge from existing SF project
>>> * Re-architect Grid based security (GSI) dependencies and adopt more
>>> general purpose security implementations.
>>> * Make sure Cloud (including hadoop) support is more first class.
>>> * Aim to have the first Apache release within the first 6 months
>>> * Verify with Apache Legal that some of the more esoteric licences in
>>> our dependencies are acceptable, or replace them as appropriate
>>>
>>> == Current Status ==
>>>
>>> The proposed tools are currently hosted on SourceForge at
>>> http://sourceforge.net/projects/ogce/ (source at
>>> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/) and are
>>> described at http://www.collab-ogce.org.
>>>
>>> == Meritocracy ==
>>>
>>> A significant portion of initial committers are already ASF
>>> Committers/Members,
>>> and the entire team is well experienced with open source software
>>> development. The existing code base has resulted from
>>> multi-institutional collaborative projects. The developers are well
>>> aware of the Apache way and will honor the meritocracy policy of ASF
>>> foundation.
>>>
>>> == Community ==
>>>
>>> To date our focus has been serving our immediate partners needs rather
>>> than looking outwards in order to build a broader community with
>>> diverse needs. Whilst the core team area likely to remain focussed on
>>> the Science Gateways communities we are keen to welcome community
>>> members from other disciplines.
>>>
>>> == Core Developers ==
>>> Our core developers consist of participants from academic,
>>> not-for-profit and for-profit organisations. Many are already well
>>> versed in The Apache Way.
>>>
>>> Amongst our initial team we have one or more committers on the
>>> following Apache top level projects; axis, geronimo, synapse, ws,
>>> ws-pmc, ws-woden as well as Apache Rave (Incubating).
>>>
>>> == Alignment ==
>>> Airavata software is built upon Apache Projects like Axis2, ODE,
>>> Rampart, Tomcat and Maven. We will try to closely align the project
>>> with ODE to ensure BPEL workflow compatibility. We will align with
>>> metadata management projects like Apache OODT. Web interfaces within
>>> the Airavata software will be synergistically developed with Apache
>>> Rave.
>>>
>>> == Known Risks ==
>>> === Orphaned products ===
>>> We acknowledge the need to seek project contributions outside the current
>>> developers. The core team actively travels and conducts workshops and
>>> tutorials at relevant academic conferences like Supercomputing, TeraGrid,
>>> Collaborative Technologies Systems and SciDAC. Previous experiences
>>> have showed that these tutorials and outreach efforts will bring in
>>> community participation. The general strategy will be to encourage
>>> users to be active in the community and develop patches and
>>> contribute. Also, the core developers use the Airavata software in
>>> multiple projects with a life span ranging from 2 to 10 years, so the
>>> risk of orphaned products is very minimal.
>>>
>>> Furthermore, by opening our doors to non-academic organisations
>>> already adopting large scale computation related projects in the ASF
>>> we hope to be able to build community beyond the proposing teams
>>> Science Gateway interests.
>>>
>>> === Inexperience with Open Source ===
>>> The core team is very familiar with open source practices. The
>>> developers include existing Apache members who have long term experience
>>> with
>>> the Apache Way. The OGCE project has been an active
>>> open source project in SourceForge since November 2006. We welcome the
>>> new directions and are well prepared to follow the Apache way.
>>>
>>> === Homogenous Developers ===
>>> We have a semi-distributed development environment distributed among
>>> Indiana University and Lanka Software Foundation. We fully expect
>>> contributions from the partnering science gateways adding to the
>>> heterogeneous development.
>>>
>>> === Reliance on Salaried Developers ===
>>> The core developers are self motivated on the project and also are
>>> funded through various federal, state and endowment research
>>> grants. Participation in these research efforts based on Airavata
>>> software is mostly voluntary and above and beyond the requirements of
>>> the salaried jobs.
>>>
>>> The Open Gateway Computing project, from which the initial code
>>> donation is sourced, is funded for the next 3 years and is mandated by
>>> the funding guidelines to open source software development -
>>> http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=1032742. We
>>> believe in the Airavata software capabilities and its vital role in
>>> providing sustainable middleware for Science Gateways. Nevertheless,
>>> the core team will actively build upon Airavata software and foster
>>> developer community outside the current core.
>>>
>>> === Relationships with Other Apache Products ===
>>> See “Alignment† above. Airavata is based on the concepts of Service
>>> Oriented Architecture and all services run within Tomcat
>>> container. The web services are based on Axis2. The orchestration of
>>> the scientific workflows uses Orchestration Director Engine. The
>>> software is built using Apache Maven.
>>>
>>> === An Excessive Fascination with the Apache Brand ===
>>> The Apache brand would certainly help promote the software suite, but
>>> gaining the brand is not the motivation for this project. Airavata is
>>> being proposed to Apache because of the belief in Apache’s meritocracy
>>> model for mentored, community-driven, open source software is the best
>>> way to develop sustainable software. See “Rational† above. Most
>>> importantly, The Apache Software Foundation will help us create an
>>> institution-neutral contribution venue and will help us build a
>>> long-standing community around Airavata to sustain and improve it
>>> beyond the span of specific, targeted research grants.
>>>
>>> == Documentation ==
>>> Existing documentation is available from the OGCE wiki,
>>> http://www.collab-ogce.org/ogce/index.php/Main_Page. In addition,
>>> there is abundance of presentation and self guided video tutorial
>>> material. Effort will be put in to collect all this information into
>>> meaningful documentation on the Apache websites.
>>>
>>> == Initial Source ==
>>> The initial source of the project is in SourceForge. The source is
>>> available for anonymous check out from svn at
>>> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/
>>>
>>> == Source and Intellectual Property Submission Plan ==
>>> Indiana University is the current holder of Intellectual Property
>>> rights for the software. The university has approved the code donation
>>> and signed trustees approval, Corporate Contributor Licence Agreement
>>> and Software Grant Agreement have been emailed to ASF secretary and
>>> received acknowledgement.
>>>
>>> Specifically Indiana University will donate 4 components into Airavata
>>> project.
>>>
>>> 1. XBaya Scientific Workflow Suite - includes a GUI for workflow
>>> composition and monitoring. The composed workflow can be exported to
>>> various workflow languages like BPEL, SCUFL, Condor DAG, Jython and
>>> Java. The defacto workflow enacting engine used is Apache ODE.
>>> 2. GFac - an application wrapper service that can be used to wrap
>>> command line-driven science applications and make them into robust,
>>> network- accessible services. This component is build on Axis2 web
>>> service stack.
>>> 3. XRegistry - a registry service for storing deployment information
>>> about wrapped application services and constructed workflows.
>>> 4. WS-Messenger - a “publish-subscribe† based message broker
>>> implemented on top of Apache Axis2 web services stack. It implements the
>>> WS-Eventing and WS-Notifications specifications and incorporates a
>>> message box component that facilities communications with clients behind
>>> firewalls and overcomes network glitches.
>>>
>>> == External Dependencies ==
>>>
>>> Following the guideline -http://www.apache.org/legal/resolved.html, the
>>> following are the dependent software and all of them are in binary
>>> format in java archive (jar files).
>>>
>>> * CDDL license - Javax activation, JSR311, Portlet-API, Servlet-API
>>> * Apache V2: cog-jglobus, globus, caster, gridsphere, Woodstox, xmpp,
>>> xsul, sigiri, atomixmiser, weps-beans.
>>> * BSD: puretls,
>>> * MIT: bcporv, hsqldb, dom4j, slf4j
>>> * PSFL: Jython
>>> * GPL 2.0: mysql-connector-java
>>> * Other:
>>> * cryptix32, cryptix-asn1 (http://www.cryptix.org/LICENSE.TXT)
>>> * backport (public domain)
>>> * jaxen (http://jaxen.codehaus.org/license.html)
>>>
>>> Licence incompatibilities (GPL) will be resolved during incubation.
>>>
>>> == Cryptography ==
>>> The software does not implement any cryptographic algorithms. However,
>>> to perform secured messaging and data movement and SSL communications,
>>> the software depends upon third party security libraries. These
>>> external libraries depend in turn on Java Security, Puretls, Cryptix
>>> and Bounce Castle libraries. Apache Cryptographic steps will be
>>> followed to register the use of these libraries.
>>>
>>> == Required Resources ==
>>>
>>> === Mailing lists ===
>>> 1. airavata-dev
>>> 2. airavata-commits
>>> 3. airavata-private
>>>
>>> === Subversion Directory ===
>>> https://svn.apache.org/repos/asf/incubator/airavata
>>>
>>> === Issue Tracking ===
>>> We intend to make use of Jira for issue tracking. Proposed key: AIRAVATA
>>>
>>> === Other Resources ===
>>>
>>> We intend to manage our website using the Apache CMS.
>>>
>>> == Initial Committers ==
>>> Names of initial committers with affiliation and current ASF status:
>>>
>>> || '''Name''' || '''Email''' || '''Affiliation''' || '''ICLA''' ||
>>> '''ASF Status''' || '''Apache Id''' ||
>>> || Suresh Marru || smarru@cs.indiana.edu || Indiana University || On
>>> File || Apache Commiter || smarru ||
>>> || Marlon Pierce || mpierce@cs.indiana.edu || Indiana University || On
>>> File || Apache Commiter || mpierce ||
>>> || Srinath Perera || hemapani@apache.org || Lanka Software Foundation ||
>>> On File || Apache Member || hemapani ||
>>> || Aleksander Slominski || aslom at us.ibm.com || IBM || On File ||
>>> Apache Member || aslom ||
>>> || Raminderjeet Singh || ramifnu@indiana.edu || Indiana University || On
>>> File || Apache Commiter || raminder ||
>>> || Archit Kulshrestha || akulshre@indiana.edu || Indiana University ||
>>> On File || N/A || N/A ||
>>> || Chathura Herath || chathura@apache.org || Indiana University || On
>>> File || Apache Commiter || chathura ||
>>> || Eran Chinthaka || chinthaka@apache.org || Indiana University || On
>>> File || Apache Member || chinthaka ||
>>> || Thilina Gunaratne || thilina@apache.org || Indiana University || On
>>> File || Apache Commiter || thilina ||
>>> || Wathsala Vithanage || wathsala@opensource.lk || Lanka Software
>>> Foundation || On File || N/A || N/A ||
>>>
>>> All the parties are affiliated with companies and organizations that
>>> are familiar with the development of open source. We expect that the
>>> amount of volunteer work will increase, and more developers will come
>>> on board.
>>>
>>> == Champion ==
>>> Ross Gardler, Apache Software Foundation
>>>
>>> == Nominated Mentors ==
>>> * Ross Gardler, Member, Apache Software Foundation
>>> * Alek Slominski, Member, Apache Software Foundation
>>> * Ate Douma, Member, Apache Software Foundation
>>> * Sanjiva Weerawarna, Member, Apache Software Foundation
>>> * Paul Fremantle, Member, Apache Software Foundation
>>>
>>> == Sponsoring Entity ==
>>> Apache Incubator Project.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>>
>
>
>
> --
> ============================
> Srinath Perera, Ph.D.
>   Senior Software Architect, WSO2 Inc.
>   Visiting Faculty, University of Moratuwa
>   Member, Apache Software Foundation
>   Research Scientist, Lanka Software Foundation
>   Blog: http://srinathsview.blogspot.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>



-- 
Chathura Herath
https://www.cs.indiana.edu/~cherath/
http://chathurah.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message