Return-Path: X-Original-To: apmail-incubator-general-archive@www.apache.org Delivered-To: apmail-incubator-general-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3CB9A3A19 for ; Thu, 28 Apr 2011 22:21:32 +0000 (UTC) Received: (qmail 82293 invoked by uid 500); 28 Apr 2011 22:21:31 -0000 Delivered-To: apmail-incubator-general-archive@incubator.apache.org Received: (qmail 82121 invoked by uid 500); 28 Apr 2011 22:21:31 -0000 Mailing-List: contact general-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@incubator.apache.org Delivered-To: mailing list general@incubator.apache.org Received: (qmail 82113 invoked by uid 99); 28 Apr 2011 22:21:31 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Apr 2011 22:21:31 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [67.19.18.210] (HELO aalto.aaltohost.com) (67.19.18.210) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Apr 2011 22:21:24 +0000 Received: from Air.local (pool-173-48-173-230.bstnma.fios.verizon.net [173.48.173.230]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by aalto.aaltohost.com (Postfix) with ESMTP id D027F488F7A for ; Thu, 28 Apr 2011 18:21:02 -0400 (EDT) Message-ID: <4DB9E84D.9030502@thoughtcraft.com> Date: Thu, 28 Apr 2011 18:21:01 -0400 From: Glen Daniels User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.15) Gecko/20110303 Lightning/1.0b2 Thunderbird/3.1.9 MIME-Version: 1.0 To: general@incubator.apache.org Subject: Re: [PROPOSAL] Airavata for the incubator References: <4DB34A51.9010005@apache.org> In-Reply-To: <4DB34A51.9010005@apache.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Checked: Checked by ClamAV on apache.org Hi Ross, all, I think in general this looks great and I'd definitely +1 bringing Airavata to the Incubator. The only point I'd explore a bit is the WS-Messenger component, in particular whether this bit might (either now or eventually) want to end up in the Axis project, since it seems like a worthy replacement to the not-very-well-maintained Savan (the WS-Eventing-over-Axis2 module in Axis). While it's certainly not necessary to have all Axis2 modules in the same place - there are plenty at WSO2 and elsewhere - it is pretty nice for both users and developers to have some of the core ones there. For users, it makes them easy to find, and for developers, it makes it easier to test everything together and ensure that Axis2 changes don't break any functionality in the extension modules. Since a bunch of the Airavata team are already Axis committers, I believe it should be possible to simply check WS-Messenger in there if the team (and the Axis team) agrees that would be a good home. Alternately Airavata could go through the incubator as-is and make a decision about this during graduation (at the risk of package-naming challenges). Cool stuff - I remember Eran talking about this a couple of years back. Thanks, --Glen On 4/23/11 5:53 PM, Ross Gardler wrote: > I would like to propose Airavata for entry into the Apache Incubator. > > The full proposal can be found at [1] and is copied at the end of this > mail. For those in a hurry here's a quick summary: > > Airavata is a software toolkit currently used to build science gateways > but that has a much wider potential use. It provides features to > compose, manage, execute, and monitor large scale applications and > workflows on computational resources ranging from local clusters to > national grids and computing clouds. Users can use Airavata back end > services and build gadgets to deploy in open social containers such as > Apache Rave and modify them to suite their needs. Airavata builds on > general concepts of service oriented computing, distributed messaging, > and workflow composition and orchestration. > > Airavata will provide web interfaces and scalable Service Oriented > Architecture based backend services to build or enhance Science Gateway > (see https://www.teragrid.org/web/science-gateways/) and similar > environments. Airavata will specifically focus on: > > 1. sophisticated server-side tools for registering and managing large > scale applications on computational resources. > > 2. graphical user interfaces to construct, execute, control, manage and > reuse of scientific workflows. > > 3. interfacing and interoperability with with various external (third > party) data and provenance management tools > > The project team consists of a number of existing Apache Committers and > the code comes from the same stable as some of the code donated to > Apache Rave (Incubating). > > We welcome your questions, suggestions, observations and support. > > Ross > > [1] http://wiki.apache.org/incubator/AiravataProposal > > FULL PROPOSAL TEXT > ================== > > = Airavata Proposal for Apache Incubator = > > == Abstract == > Airavata is a software toolkit currently used to build science > gateways but that has a much wider potential use. It provides features > to compose, manage, execute, and monitor large scale applications and > workflows on computational resources ranging from local clusters to > national grids and computing clouds. Users can use Airavata back end > services and build gadgets to deploy in open social containers such as > Apache Rave and modify them to suite their needs. Airavata builds on > general concepts of service oriented computing, distributed messaging, > and workflow composition and orchestration. > > > == Proposal == > > Airavata will provide web interfaces and scalable Service Oriented > Architecture based backend services to build or enhance Science > Gateway (see https://www.teragrid.org/web/science-gateways/) > and similar environments. Airavata will specifically focus on: > > 1. sophisticated server-side tools for registering and managing large > scale applications on computational resources. > 2. graphical user interfaces to construct, execute, control, manage and > reuse of scientific workflows. > 3. interfacing and interoperability with with various external (third > party) data and provenance management tools. > > == Background == > Working in close quarters with Apache Axis2 committers and inspired by > the true open source community driven software development of ASF, > Suresh Marru and Marlon Pierce have been pioneering the idea of a > Science Gateways software-based Apache project since late 2008. Many > Apache members have fostered these ideas and guided them to arrive at > this proposal. > > Currently the software is a actively used in various science > gateways. But the tools are general purpose and build upon widely used > Apache tools like Axis2, ODE engine. The core team is motivated to > expand the community and build a community welcoming both synergistic > software components and also new usage scenarios. > > It is perhaps worth noting that one of the three seed projects that > make up the Apache Rave (Incubating) project is also the product of > this same team and is derived from the same Science Gateways > community. > > == Rationale == > > The nature of computational problems has evolved from simple desktop > calculations to complex, multidisciplinary activities that require the > monitoring and analysis of remote data streams, database and web > search and large ensembles of simulations. In the academic domain > Science Gateways have emerged to address these needs and have built > software platforms that provide a community of users with the ability > to easily solve computational problems within a specific domain. The > tools developed to support these gateways are potentially of value to > any organisation needing to perform complex computations. Gateways > provide a convenient interface to the underlying infrastrucure without > the need for a deep understanding of the intricacies that > infrastructure. > > We summarize the rationale for choosing The Apache Software Foundation > (ASF) below. This is what we hope to gain from participating in the > ASF. > > 1. '''Broader impact''': our science gateway tool set is based on > Service Oriented Architecture principles, and it has always been our > goal to align our software with broader trends in the development of > software for distributed systems. Participating in the ASF provides a > concrete way to implement this idea. In particular, we have done > extensive work on the workflow systems, messaging, and application > management as Web services from the perspective of computational science > use cases (i.e., high failure rates, very long running jobs, dynamic > service creation, workflows not expressible as directed acyclic graphs, > etc). These requirements and our work to implement them have already had > direct impact on the Apache Axis 2 and Apache ODE projects. As an Apache > project, it is hoped that our community will have an enhanced > opportunity for collaboration and complementary development with Apache > Hadoop (for scientific application management), Apache QPID (for > messaging), Apache Rave (incubator - Open Social Container) and others. > It is our goal to expand our software’s usage beyond just science > gateways to the broader enterprise community. > 2. '''Sustainability''': Science gateway software development (and > cyberinfrastructure software generally) is primarily funded in the US by > the National Science Foundation (NSF), so the long term sustainability > of software across funding cycles is a longstanding problem. The NSF is > attempting to solve this problem, and its vision for sustainable > software is described here: > http://www.nsf.gov/pubs/2010/nsf10015/nsf10015.jsp. Participating in the > ASF is our project’s vision for reaching software sustainability that > underpins the NSF CF21 vision. As a successful ASF project (after > incubation), we will have created a community led, rather than funding > led, environment for the development of our sotware. This community, > through our community engagement work and adoption of meritocratic > principles, will expand beyond our current core team and existing > project collaborations. This will greatly increase the chances that our > software will continue to grow and improve beyond the participation of > any individuals. > 3. '''Maturity''': much of the software included in this proposal was > developed initially by graduate students as part of their Ph. D. work. > The Open Grid Computing Environment has devoted significant effort > (through salaried staff and volunteers from collaborating institutions) > to convert these research projects into mature, reliable, well-written, > packaged components. The code is currently hosted at SourceForge, but we > recognize the need to go beyond just the SourceForge support tools to > participate in a real community of software engineering experts. It is > our desire, through the Apache Incubator, to take our software > engineering efforts to a higher level by learning from the substantial > experience of appropraite Apache Committers. Apache mentors will provide > initial guidance, as will the attraction of additional committers from > the relevant Apache projects. > > == Initial Goals == > > * Implement a standalone version of the code base with a simple hello > world service, workflow and gadget(s) to access the examples. > * Migration of documentation and design knowledge from existing SF project > * Re-architect Grid based security (GSI) dependencies and adopt more > general purpose security implementations. > * Make sure Cloud (including hadoop) support is more first class. > * Aim to have the first Apache release within the first 6 months > * Verify with Apache Legal that some of the more esoteric licences in > our dependencies are acceptable, or replace them as appropriate > > == Current Status == > > The proposed tools are currently hosted on SourceForge at > http://sourceforge.net/projects/ogce/ (source at > https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/) and are > described at http://www.collab-ogce.org. > > == Meritocracy == > > A significant portion of initial committers are already ASF > Committers/Members, > and the entire team is well experienced with open source software > development. The existing code base has resulted from > multi-institutional collaborative projects. The developers are well > aware of the Apache way and will honor the meritocracy policy of ASF > foundation. > > == Community == > > To date our focus has been serving our immediate partners needs rather > than looking outwards in order to build a broader community with > diverse needs. Whilst the core team area likely to remain focussed on > the Science Gateways communities we are keen to welcome community > members from other disciplines. > > == Core Developers == > Our core developers consist of participants from academic, > not-for-profit and for-profit organisations. Many are already well > versed in The Apache Way. > > Amongst our initial team we have one or more committers on the > following Apache top level projects; axis, geronimo, synapse, ws, > ws-pmc, ws-woden as well as Apache Rave (Incubating). > > == Alignment == > Airavata software is built upon Apache Projects like Axis2, ODE, > Rampart, Tomcat and Maven. We will try to closely align the project > with ODE to ensure BPEL workflow compatibility. We will align with > metadata management projects like Apache OODT. Web interfaces within > the Airavata software will be synergistically developed with Apache > Rave. > > == Known Risks == > === Orphaned products === > We acknowledge the need to seek project contributions outside the current > developers. The core team actively travels and conducts workshops and > tutorials at relevant academic conferences like Supercomputing, TeraGrid, > Collaborative Technologies Systems and SciDAC. Previous experiences > have showed that these tutorials and outreach efforts will bring in > community participation. The general strategy will be to encourage > users to be active in the community and develop patches and > contribute. Also, the core developers use the Airavata software in > multiple projects with a life span ranging from 2 to 10 years, so the > risk of orphaned products is very minimal. > > Furthermore, by opening our doors to non-academic organisations > already adopting large scale computation related projects in the ASF > we hope to be able to build community beyond the proposing teams > Science Gateway interests. > > === Inexperience with Open Source === > The core team is very familiar with open source practices. The > developers include existing Apache members who have long term experience > with > the Apache Way. The OGCE project has been an active > open source project in SourceForge since November 2006. We welcome the > new directions and are well prepared to follow the Apache way. > > === Homogenous Developers === > We have a semi-distributed development environment distributed among > Indiana University and Lanka Software Foundation. We fully expect > contributions from the partnering science gateways adding to the > heterogeneous development. > > === Reliance on Salaried Developers === > The core developers are self motivated on the project and also are > funded through various federal, state and endowment research > grants. Participation in these research efforts based on Airavata > software is mostly voluntary and above and beyond the requirements of > the salaried jobs. > > The Open Gateway Computing project, from which the initial code > donation is sourced, is funded for the next 3 years and is mandated by > the funding guidelines to open source software development - > http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=1032742. We > believe in the Airavata software capabilities and its vital role in > providing sustainable middleware for Science Gateways. Nevertheless, > the core team will actively build upon Airavata software and foster > developer community outside the current core. > > === Relationships with Other Apache Products === > See “Alignment” above. Airavata is based on the concepts of Service > Oriented Architecture and all services run within Tomcat > container. The web services are based on Axis2. The orchestration of > the scientific workflows uses Orchestration Director Engine. The > software is built using Apache Maven. > > === An Excessive Fascination with the Apache Brand === > The Apache brand would certainly help promote the software suite, but > gaining the brand is not the motivation for this project. Airavata is > being proposed to Apache because of the belief in Apache’s meritocracy > model for mentored, community-driven, open source software is the best > way to develop sustainable software. See “Rational” above. Most > importantly, The Apache Software Foundation will help us create an > institution-neutral contribution venue and will help us build a > long-standing community around Airavata to sustain and improve it > beyond the span of specific, targeted research grants. > > == Documentation == > Existing documentation is available from the OGCE wiki, > http://www.collab-ogce.org/ogce/index.php/Main_Page. In addition, > there is abundance of presentation and self guided video tutorial > material. Effort will be put in to collect all this information into > meaningful documentation on the Apache websites. > > == Initial Source == > The initial source of the project is in SourceForge. The source is > available for anonymous check out from svn at > https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/ > > == Source and Intellectual Property Submission Plan == > Indiana University is the current holder of Intellectual Property > rights for the software. The university has approved the code donation > and signed trustees approval, Corporate Contributor Licence Agreement > and Software Grant Agreement have been emailed to ASF secretary and > received acknowledgement. > > Specifically Indiana University will donate 4 components into Airavata > project. > > 1. XBaya Scientific Workflow Suite - includes a GUI for workflow > composition and monitoring. The composed workflow can be exported to > various workflow languages like BPEL, SCUFL, Condor DAG, Jython and > Java. The defacto workflow enacting engine used is Apache ODE. > 2. GFac - an application wrapper service that can be used to wrap > command line-driven science applications and make them into robust, > network- accessible services. This component is build on Axis2 web > service stack. > 3. XRegistry - a registry service for storing deployment information > about wrapped application services and constructed workflows. > 4. WS-Messenger - a “publish-subscribe” based message broker > implemented on top of Apache Axis2 web services stack. It implements the > WS-Eventing and WS-Notifications specifications and incorporates a > message box component that facilities communications with clients behind > firewalls and overcomes network glitches. > > == External Dependencies == > > Following the guideline -http://www.apache.org/legal/resolved.html, the > following are the dependent software and all of them are in binary > format in java archive (jar files). > > * CDDL license - Javax activation, JSR311, Portlet-API, Servlet-API > * Apache V2: cog-jglobus, globus, caster, gridsphere, Woodstox, xmpp, > xsul, sigiri, atomixmiser, weps-beans. > * BSD: puretls, > * MIT: bcporv, hsqldb, dom4j, slf4j > * PSFL: Jython > * GPL 2.0: mysql-connector-java > * Other: > * cryptix32, cryptix-asn1 (http://www.cryptix.org/LICENSE.TXT) > * backport (public domain) > * jaxen (http://jaxen.codehaus.org/license.html) > > Licence incompatibilities (GPL) will be resolved during incubation. > > == Cryptography == > The software does not implement any cryptographic algorithms. However, > to perform secured messaging and data movement and SSL communications, > the software depends upon third party security libraries. These > external libraries depend in turn on Java Security, Puretls, Cryptix > and Bounce Castle libraries. Apache Cryptographic steps will be > followed to register the use of these libraries. > > == Required Resources == > > === Mailing lists === > 1. airavata-dev > 2. airavata-commits > 3. airavata-private > > === Subversion Directory === > https://svn.apache.org/repos/asf/incubator/airavata > > === Issue Tracking === > We intend to make use of Jira for issue tracking. Proposed key: AIRAVATA > > === Other Resources === > > We intend to manage our website using the Apache CMS. > > == Initial Committers == > Names of initial committers with affiliation and current ASF status: > > || '''Name''' || '''Email''' || '''Affiliation''' || '''ICLA''' || > '''ASF Status''' || '''Apache Id''' || > || Suresh Marru || smarru@cs.indiana.edu || Indiana University || On > File || Apache Commiter || smarru || > || Marlon Pierce || mpierce@cs.indiana.edu || Indiana University || On > File || Apache Commiter || mpierce || > || Srinath Perera || hemapani@apache.org || Lanka Software Foundation || > On File || Apache Member || hemapani || > || Aleksander Slominski || aslom at us.ibm.com || IBM || On File || > Apache Member || aslom || > || Raminderjeet Singh || ramifnu@indiana.edu || Indiana University || On > File || Apache Commiter || raminder || > || Archit Kulshrestha || akulshre@indiana.edu || Indiana University || > On File || N/A || N/A || > || Chathura Herath || chathura@apache.org || Indiana University || On > File || Apache Commiter || chathura || > || Eran Chinthaka || chinthaka@apache.org || Indiana University || On > File || Apache Member || chinthaka || > || Thilina Gunaratne || thilina@apache.org || Indiana University || On > File || Apache Commiter || thilina || > || Wathsala Vithanage || wathsala@opensource.lk || Lanka Software > Foundation || On File || N/A || N/A || > > All the parties are affiliated with companies and organizations that > are familiar with the development of open source. We expect that the > amount of volunteer work will increase, and more developers will come > on board. > > == Champion == > Ross Gardler, Apache Software Foundation > > == Nominated Mentors == > * Ross Gardler, Member, Apache Software Foundation > * Alek Slominski, Member, Apache Software Foundation > * Ate Douma, Member, Apache Software Foundation > * Sanjiva Weerawarna, Member, Apache Software Foundation > * Paul Fremantle, Member, Apache Software Foundation > > == Sponsoring Entity == > Apache Incubator Project. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org > For additional commands, e-mail: general-help@incubator.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org For additional commands, e-mail: general-help@incubator.apache.org