Mailing-List: contact general-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@incubator.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Subject: Re: [PROPOSAL] Airavata for the incubator
References: <4DB34A51.9010005@apache.org>
 <BANLkTimELFqQMShCB=XJDoqsTuCHqVaeCg@mail.gmail.com>
From: Ross Gardler <rgardler@apache.org>
Content-Type: text/plain;
	charset=utf-8
In-Reply-To: <BANLkTimELFqQMShCB=XJDoqsTuCHqVaeCg@mail.gmail.com>
Message-Id: <5D7E24B7-2A84-4D76-AC0E-E9D9332D08DA@apache.org>
Date: Thu, 28 Apr 2011 21:10:21 +0100
To: "general@incubator.apache.org" <general@incubator.apache.org>
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (iPhone Mail 8H7)


On 28 Apr 2011, at 20:45, Sagara Gunathunga <sagara@apache.org> wrote:

> Hi Ross,
>=20
> +1 and  I would like to contribute for this project.

Excellent. We look forward to working with you.=20

Ross


>=20
> Thanks !
>=20
> On Sun, Apr 24, 2011 at 3:23 AM, Ross Gardler <rgardler@apache.org> wrote:=

>> I would like to propose Airavata for entry into the Apache Incubator.
>>=20
>> The full proposal can be found at [1] and is copied at the end of this ma=
il.
>> For those in a hurry here's a quick summary:
>>=20
>> Airavata is a software toolkit currently used to build science gateways b=
ut
>> that has a much wider potential use. It provides features to compose,
>> manage, execute, and monitor large scale applications and workflows on
>> computational resources ranging from local clusters to national grids and=

>> computing clouds. Users can use Airavata back end services and build gadg=
ets
>> to deploy in open social containers such as Apache Rave and modify them t=
o
>> suite their needs. Airavata builds on general concepts of service oriente=
d
>> computing, distributed messaging, and workflow composition and
>> orchestration.
>>=20
>> Airavata will provide web interfaces and scalable Service Oriented
>> Architecture based backend services to build or enhance Science Gateway (=
see
>> https://www.teragrid.org/web/science-gateways/) and similar environments.=

>> Airavata will specifically focus on:
>>=20
>> 1.  sophisticated server-side tools for registering and managing large sc=
ale
>> applications on computational resources.
>>=20
>> 2.  graphical user interfaces to construct, execute, control, manage and
>> reuse of scientific workflows.
>>=20
>> 3.  interfacing and interoperability with with various external (third
>> party) data and provenance management tools
>>=20
>> The project team consists of a number of existing Apache Committers and t=
he
>> code comes from the same stable as some of the code donated to Apache Rav=
e
>> (Incubating).
>>=20
>> We welcome your questions, suggestions, observations and support.
>>=20
>> Ross
>>=20
>> [1] http://wiki.apache.org/incubator/AiravataProposal
>>=20
>> FULL PROPOSAL TEXT
>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>>=20
>> =3D Airavata Proposal for Apache Incubator =3D
>>=20
>> =3D=3D Abstract =3D=3D
>> Airavata is a software toolkit currently used to build science
>> gateways but that has a much wider potential use. It provides features
>> to compose, manage, execute, and monitor large scale applications and
>> workflows on computational resources ranging from local clusters to
>> national grids and computing clouds. Users can use Airavata back end
>> services and build gadgets to deploy in open social containers such as
>> Apache Rave and modify them to suite their needs. Airavata builds on
>> general concepts of service oriented computing, distributed messaging,
>> and workflow composition and orchestration.
>>=20
>>=20
>> =3D=3D Proposal =3D=3D
>>=20
>> Airavata will provide web interfaces and scalable Service Oriented
>> Architecture based backend services to build or enhance Science
>> Gateway (see https://www.teragrid.org/web/science-gateways/)
>> and similar environments. Airavata will specifically focus on:
>>=20
>>  1. sophisticated server-side tools for registering and managing large sc=
ale
>> applications on computational resources.
>>  2. graphical user interfaces to construct, execute, control, manage and
>> reuse of scientific workflows.
>>  3. interfacing and interoperability with with various external (third
>> party) data and provenance management tools.
>>=20
>> =3D=3D Background =3D=3D
>> Working in close quarters with Apache Axis2 committers and inspired by
>> the true open source community driven software development of ASF,
>> Suresh Marru and Marlon Pierce have been pioneering the idea of a
>> Science Gateways software-based Apache project since late 2008. Many
>> Apache members have fostered these ideas and guided them to arrive at
>> this proposal.
>>=20
>> Currently the software is a actively used in various science
>> gateways. But the tools are general purpose and build upon widely used
>> Apache tools like Axis2, ODE engine. The core team is motivated to
>> expand the community and build a community welcoming both synergistic
>> software components and also new usage scenarios.
>>=20
>> It is perhaps worth noting that one of the three seed projects that
>> make up the Apache Rave (Incubating) project is also the product of
>> this same team and is derived from the same Science Gateways
>> community.
>>=20
>> =3D=3D Rationale =3D=3D
>>=20
>> The nature of computational problems has evolved from simple desktop
>> calculations to complex, multidisciplinary activities that require the
>> monitoring and analysis of remote data streams, database and web
>> search and large ensembles of simulations. In the academic domain
>> Science Gateways have emerged to address these needs and have built
>> software platforms that provide a community of users with the ability
>> to easily solve computational problems within a specific domain. The
>> tools developed to support these gateways are potentially of value to
>> any organisation needing to perform complex computations. Gateways
>> provide a convenient interface to the underlying infrastrucure without
>> the need for a deep understanding of the intricacies that
>> infrastructure.
>>=20
>> We summarize the rationale for choosing The Apache Software Foundation
>> (ASF) below. This is what we hope to gain from participating in the
>> ASF.
>>=20
>>  1. '''Broader impact''': our science gateway tool set is based on Servic=
e
>> Oriented Architecture principles, and it has always been our goal to alig=
n
>> our software with broader trends in the development of software for
>> distributed systems.  Participating in the ASF provides a concrete way to=

>> implement this idea.  In particular, we have done extensive work on the
>> workflow systems, messaging, and application management as Web services f=
rom
>> the perspective of computational science use cases (i.e., high failure
>> rates, very long running jobs, dynamic service creation, workflows not
>> expressible as directed acyclic graphs, etc). These requirements and our
>> work to implement them have already had direct impact on the Apache Axis 2=

>> and Apache ODE projects. As an Apache project, it is hoped that our
>> community will have an enhanced opportunity for collaboration and
>> complementary development with Apache Hadoop (for scientific application
>> management), Apache QPID (for messaging), Apache Rave (incubator - Open
>> Social Container) and others.  It is our goal to expand our software=C3=A2=
=E2=82=AC=E2=84=A2s
>> usage beyond just science gateways to the broader enterprise community.
>>  2. '''Sustainability''': Science gateway software development (and
>> cyberinfrastructure software generally) is primarily funded in the US by t=
he
>> National Science Foundation (NSF), so the long term sustainability of
>> software across funding cycles is a longstanding problem.  The NSF is
>> attempting to solve this problem, and its vision for sustainable software=
 is
>> described  here: http://www.nsf.gov/pubs/2010/nsf10015/nsf10015.jsp.
>> Participating in the ASF is our project=C3=A2=E2=82=AC=E2=84=A2s vision f=
or reaching software
>> sustainability that underpins the NSF CF21 vision.  As a successful ASF
>> project (after incubation), we will have created a community led, rather
>> than funding led, environment for the development of our sotware. This
>> community, through our community engagement work and adoption of
>> meritocratic principles, will expand beyond our current core team and
>> existing project collaborations.  This will greatly increase the chances
>> that our software will continue to grow and improve beyond the participat=
ion
>> of any individuals.
>>  3. '''Maturity''': much of the software included in this proposal was
>> developed initially by graduate students as part of their Ph. D. work. Th=
e
>> Open Grid Computing Environment has devoted significant effort (through
>> salaried staff and volunteers from collaborating institutions) to convert=

>> these research projects into mature, reliable, well-written, packaged
>> components.  The code is currently hosted at SourceForge, but we recogniz=
e
>> the need to go beyond just the SourceForge support tools to participate i=
n a
>> real community of software engineering experts. It is our desire, through=

>> the Apache Incubator, to take our software engineering efforts to a highe=
r
>> level by learning from the substantial experience of appropraite Apache
>> Committers. Apache mentors will provide initial guidance, as will  the
>> attraction of additional committers from the relevant Apache projects.
>>=20
>> =3D=3D Initial Goals =3D=3D
>>=20
>>  * Implement a standalone version of the code base with a simple hello wo=
rld
>> service, workflow and gadget(s) to access the examples.
>>  * Migration of documentation and design knowledge from existing SF proje=
ct
>>  * Re-architect Grid based security (GSI) dependencies and adopt more
>> general purpose security implementations.
>>  * Make sure Cloud (including hadoop) support is more first class.
>>  * Aim to have the first Apache release within the first 6 months
>>  * Verify with Apache Legal that some of the more esoteric licences in ou=
r
>> dependencies are acceptable, or replace them as appropriate
>>=20
>> =3D=3D Current Status =3D=3D
>>=20
>> The proposed tools are currently hosted on SourceForge at
>> http://sourceforge.net/projects/ogce/ (source at
>> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/) and are
>> described at http://www.collab-ogce.org.
>>=20
>> =3D=3D Meritocracy =3D=3D
>>=20
>> A significant portion of initial committers are already ASF
>> Committers/Members,
>> and the entire team is well experienced with open source software
>> development. The existing code base has resulted from
>> multi-institutional collaborative projects. The developers are well
>> aware of the Apache way and will honor the meritocracy policy of ASF
>> foundation.
>>=20
>> =3D=3D Community =3D=3D
>>=20
>> To date our focus has been serving our immediate partners needs rather
>> than looking outwards in order to build a broader community with
>> diverse needs. Whilst the core team area likely to remain focussed on
>> the Science Gateways communities we are keen to welcome community
>> members from other disciplines.
>>=20
>> =3D=3D Core Developers =3D=3D
>> Our core developers consist of participants from academic,
>> not-for-profit and for-profit organisations. Many are already well
>> versed in The Apache Way.
>>=20
>> Amongst our initial team we have one or more committers on the
>> following Apache top level projects; axis, geronimo, synapse, ws,
>> ws-pmc, ws-woden as well as Apache Rave (Incubating).
>>=20
>> =3D=3D Alignment =3D=3D
>> Airavata software is built upon Apache Projects like Axis2, ODE,
>> Rampart, Tomcat and Maven. We will try to closely align the project
>> with ODE to ensure BPEL workflow compatibility. We will align with
>> metadata management projects like Apache OODT. Web interfaces within
>> the Airavata software will be synergistically developed with Apache
>> Rave.
>>=20
>> =3D=3D Known Risks =3D=3D
>> =3D=3D=3D Orphaned products =3D=3D=3D
>> We acknowledge the need to seek project contributions outside the current=

>> developers. The core team actively travels and conducts workshops and
>> tutorials at relevant academic conferences like Supercomputing, TeraGrid,=

>> Collaborative Technologies Systems and SciDAC. Previous experiences
>> have showed that these tutorials and outreach efforts will bring in
>> community participation. The general strategy will be to encourage
>> users to be active in the community and develop patches and
>> contribute. Also, the core developers use the Airavata software in
>> multiple projects with a life span ranging from 2 to 10 years, so the
>> risk of orphaned products is very minimal.
>>=20
>> Furthermore, by opening our doors to non-academic organisations
>> already adopting large scale computation related projects in the ASF
>> we hope to be able to build community beyond the proposing teams
>> Science Gateway interests.
>>=20
>> =3D=3D=3D Inexperience with Open Source =3D=3D=3D
>> The core team is very familiar with open source practices. The
>> developers include existing Apache members who have long term experience
>> with
>> the Apache Way. The OGCE project has been an active
>> open source project in SourceForge since November 2006. We welcome the
>> new directions and are well prepared to follow the Apache way.
>>=20
>> =3D=3D=3D Homogenous Developers =3D=3D=3D
>> We have a semi-distributed development environment distributed among
>> Indiana University and Lanka Software Foundation. We fully expect
>> contributions from the partnering science gateways adding to the
>> heterogeneous development.
>>=20
>> =3D=3D=3D Reliance on Salaried Developers =3D=3D=3D
>> The core developers are self motivated on the project and also are
>> funded through various federal, state and endowment research
>> grants. Participation in these research efforts based on Airavata
>> software is mostly voluntary and above and beyond the requirements of
>> the salaried jobs.
>>=20
>> The Open Gateway Computing project, from which the initial code
>> donation is sourced, is funded for the next 3 years and is mandated by
>> the funding guidelines to open source software development -
>> http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=3D1032742. We
>> believe in the Airavata software capabilities and its vital role in
>> providing sustainable middleware for Science Gateways. Nevertheless,
>> the core team will actively build upon Airavata software and foster
>> developer community outside the current core.
>>=20
>> =3D=3D=3D Relationships with Other Apache Products =3D=3D=3D
>> See =C3=A2=E2=82=AC=C5=93Alignment=C3=A2=E2=82=AC  above.  Airavata is ba=
sed on the concepts of Service
>> Oriented Architecture and all services run within Tomcat
>> container. The web services are based on Axis2. The orchestration of
>> the scientific workflows uses Orchestration Director Engine. The
>> software is built using Apache Maven.
>>=20
>> =3D=3D=3D An Excessive Fascination with the Apache Brand =3D=3D=3D
>> The Apache brand would certainly help promote the software suite, but
>> gaining the brand is not the motivation for this project. Airavata is
>> being proposed to Apache because of the belief in Apache=C3=A2=E2=82=AC=E2=
=84=A2s meritocracy
>> model for mentored, community-driven, open source software is the best
>> way to develop sustainable software. See =C3=A2=E2=82=AC=C5=93Rational=C3=
=A2=E2=82=AC  above. Most
>> importantly, The Apache Software Foundation will help us create an
>> institution-neutral contribution venue and will help us build a
>> long-standing community around Airavata to sustain and improve it
>> beyond the span of specific, targeted research grants.
>>=20
>> =3D=3D Documentation =3D=3D
>> Existing documentation is available from the OGCE wiki,
>> http://www.collab-ogce.org/ogce/index.php/Main_Page. In addition,
>> there is abundance of presentation and self guided video tutorial
>> material. Effort will be put in to collect all this information into
>> meaningful documentation on the Apache websites.
>>=20
>> =3D=3D Initial Source =3D=3D
>> The initial source of the project is in SourceForge. The source is
>> available for anonymous check out from svn at
>> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/
>>=20
>> =3D=3D Source and Intellectual Property Submission Plan =3D=3D
>> Indiana University is the current holder of Intellectual Property
>> rights for the software. The university has approved the code donation
>> and signed trustees approval, Corporate Contributor Licence Agreement
>> and Software Grant Agreement have been emailed to ASF secretary and
>> received acknowledgement.
>>=20
>> Specifically Indiana University will donate 4 components into Airavata
>> project.
>>=20
>>  1. XBaya Scientific Workflow Suite - includes a GUI for workflow
>> composition and monitoring. The composed workflow can be exported to vari=
ous
>> workflow languages like BPEL, SCUFL, Condor DAG, Jython and Java. The
>> defacto workflow enacting engine used is Apache ODE.
>>  2. GFac - an application wrapper service that can be used to wrap comman=
d
>> line-driven science applications and make them into robust, network-
>> accessible services. This component is build on Axis2 web service stack.
>>  3. XRegistry - a registry service for storing deployment information abo=
ut
>> wrapped application services and constructed workflows.
>>  4. WS-Messenger - a =C3=A2=E2=82=AC=C5=93publish-subscribe=C3=A2=E2=82=AC=
  based message broker
>> implemented on top of Apache Axis2 web services stack. It implements the
>> WS-Eventing and WS-Notifications specifications and incorporates a messag=
e
>> box component that facilities communications with clients behind firewall=
s
>> and overcomes network glitches.
>>=20
>> =3D=3D External Dependencies =3D=3D
>>=20
>> Following the guideline -http://www.apache.org/legal/resolved.html, the
>> following are the dependent software and all of them are in binary format=
 in
>> java archive (jar files).
>>=20
>>   * CDDL license  - Javax activation, JSR311, Portlet-API, Servlet-API
>>   * Apache V2: cog-jglobus, globus, caster, gridsphere, Woodstox, xmpp,
>> xsul, sigiri, atomixmiser, weps-beans.
>>   * BSD: puretls,
>>   * MIT: bcporv, hsqldb, dom4j, slf4j
>>   * PSFL: Jython
>>   * GPL 2.0: mysql-connector-java
>>   * Other:
>>     * cryptix32, cryptix-asn1 (http://www.cryptix.org/LICENSE.TXT)
>>     * backport (public domain)
>>     * jaxen (http://jaxen.codehaus.org/license.html)
>>=20
>> Licence incompatibilities (GPL) will be resolved during incubation.
>>=20
>> =3D=3D Cryptography =3D=3D
>> The software does not implement any cryptographic algorithms. However,
>> to perform secured messaging and data movement and SSL communications,
>> the software depends upon third party security libraries. These
>> external libraries depend in turn on Java Security, Puretls, Cryptix
>> and Bounce Castle libraries. Apache Cryptographic steps will be
>> followed to register the use of these libraries.
>>=20
>> =3D=3D Required Resources =3D=3D
>>=20
>> =3D=3D=3D Mailing lists =3D=3D=3D
>>  1. airavata-dev
>>  2. airavata-commits
>>  3. airavata-private
>>=20
>> =3D=3D=3D Subversion Directory =3D=3D=3D
>> https://svn.apache.org/repos/asf/incubator/airavata
>>=20
>> =3D=3D=3D Issue Tracking =3D=3D=3D
>> We intend to make use of Jira for issue tracking. Proposed key: AIRAVATA
>>=20
>> =3D=3D=3D Other Resources =3D=3D=3D
>>=20
>> We intend to manage our website using the Apache CMS.
>>=20
>> =3D=3D Initial Committers =3D=3D
>> Names of initial committers with affiliation and current ASF status:
>>=20
>> || '''Name'''           || '''Email'''            || '''Affiliation'''
>>  || '''ICLA''' || '''ASF Status''' || '''Apache Id''' ||
>> || Suresh Marru         || smarru@cs.indiana.edu  || Indiana University
>>    || On File || Apache Commiter || smarru    ||
>> || Marlon Pierce        || mpierce@cs.indiana.edu || Indiana University
>>    || On File || Apache Commiter || mpierce   ||
>> || Srinath Perera       || hemapani@apache.org    || Lanka Software
>> Foundation || On File || Apache Member   || hemapani  ||
>> || Aleksander Slominski || aslom at us.ibm.com    || IBM        || On Fil=
e
>> || Apache Member   || aslom     ||
>> || Raminderjeet Singh   || ramifnu@indiana.edu    || Indiana University
>>    || On File || Apache Commiter || raminder  ||
>> || Archit Kulshrestha   || akulshre@indiana.edu   || Indiana University
>>    || On File || N/A             || N/A       ||
>> || Chathura Herath      || chathura@apache.org    || Indiana University
>>    || On File || Apache Commiter || chathura  ||
>> || Eran Chinthaka       || chinthaka@apache.org   || Indiana University
>>    || On File || Apache Member   || chinthaka ||
>> || Thilina Gunaratne    || thilina@apache.org     || Indiana University
>>    || On File || Apache Commiter || thilina   ||
>> || Wathsala Vithanage   || wathsala@opensource.lk || Lanka Software
>> Foundation || On File || N/A             || N/A       ||
>>=20
>> All the parties are affiliated with companies and organizations that
>> are familiar with the development of open source. We expect that the
>> amount of volunteer work will increase, and more developers will come
>> on board.
>>=20
>> =3D=3D Champion =3D=3D
>> Ross Gardler, Apache Software Foundation
>>=20
>> =3D=3D Nominated Mentors =3D=3D
>>  * Ross Gardler, Member, Apache Software Foundation
>>  * Alek Slominski, Member, Apache Software Foundation
>>  * Ate Douma, Member, Apache Software Foundation
>>  * Sanjiva Weerawarna, Member, Apache Software Foundation
>>  * Paul Fremantle, Member, Apache Software Foundation
>>=20
>> =3D=3D Sponsoring Entity =3D=3D
>> Apache Incubator Project.
>>=20
>>=20
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>=20
>>=20
>=20
>=20
>=20
> --=20
> Sagara Gunathunga
>=20
> Blog - http://ssagara.blogspot.com
> Web - http://people.apache.org/~sagara/
>=20
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>=20

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org